Audio user interface with selective audio field expansion

ABSTRACT

An audio user interface is provided in which items are represented in an audio field by corresponding synthesized sound sources from where sounds related to the items appear to emanate. The interface is generated by apparatus that includes functionality for determining for each sound source, a rendering position at which the sound source is to be synthesized to sound in the audio field. To facilitate discrimination between the synthesized sound sources, the sound-source rendering positions are adjusted such that the audio field is dilated in a region of the audio field about a focus reference; this focus reference is, for example, the current direction of facing of the user or the position of an audio cursor in the audio field.

FIELD OF THE INVENTION

The present invention relates to facilitation of clear presentation inaudio user interfaces in which sound sources are synthesised in an audiofield.

BACKGROUND OF THE INVENTION

The human auditory system, including related brain functions, is capableof localizing sounds in three dimensions notwithstanding that only twosound inputs are received (left and right ear). Research over the yearshas shown that localization in azimuth, elevation and range is dependenton a number of cues derived from the received sound. The nature of thesecues is outlined below.

Azimuth Cues—The main azimuth cues are Interaural Time Difference(ITD—sound on the right of a hearer arrives in the right ear first) andInteraural Intensity Difference (IID—sound on the right appears louderin the right ear). ITD and IIT cues are complementary inasmuch as theformer works better at low frequencies and the latter better at highfrequencies.

Elevation Cues—The primary cue for elevation depends on the acousticproperties of the outer ear or pinna. In particular, there is anelevation-dependent frequency notch in the response of the ear, thenotch frequency usually being in the range 6-16 kHz depending on theshape of the hearer's pinna. The human brain can therefore deriveelevation information based on the strength of the received sound at thepinna notch frequency, having regard to the expected signal strengthrelative to the other sound frequencies being received.

Range Cues—These include:

-   -   loudness (the nearer the source, the louder it will be; however,        to be useful, something must be known or assumed about the        source characteristics),    -   motion parallax (change in source azimuth in response to head        movement is range dependent), and    -   ratio of direct to reverberant sound (the fall-off in energy        reaching the ear as range increases is less for reverberant        sound than direct sound so that the ratio will be large for        nearby sources and small for more distant sources).

It may also be noted that in order avoid source-localization errorsarising from sound reflections, humans localize sound sources on thebasis of sounds that reach the ears first (an exception is where thedirect/reverberant ratio is used for range determination).

Getting a sound system (sound producing apparatus) to output sounds thatwill be localized by a hearer to desired locations, is not astraight-forward task and generally requires an understanding of theforegoing cues. Simple stereo sound systems with left and right speakersor headphones can readily simulate sound sources at different azimuthpositions; however, adding variations in range and elevation is muchmore complex. One known approach to producing a 3D audio field that isoften used in cinemas and theatres, is to use many loudspeakers situatedaround the listener (in practice, it is possible to use one largespeaker for the low frequency content and many small speakers for thehigh-frequency content, as the auditory system will tend to localize onthe basis of the high frequency component, this effect being known asthe Franssen effect). Such many-speaker systems are not, however,practical for most situations.

For sound sources that have a fixed presentation (non-interactive), itis possible to produce convincing 3D audio through headphones simply byrecording the sounds that would be heard at left and right eardrums werethe hearer actually present. Such recordings, known as binauralrecordings, have certain disadvantages including the need forheadphones, the lack of interactive controllability of the sourcelocation, and unreliable elevation effects due to the variation in pinnashapes between different hearers.

To enable a sound source to be variably positioned in a 3D audio field,a number of systems have evolved that are based on a transfer functionrelating source sound pressures to ear drum sound pressures. Thistransfer function is known as the Head Related Transfer Function (HRTF)and the associated impulse response, as the Head Related ImpulseResponse (HRIR). If the HRTF is known for the left and right ears,binaural signals can be synthesized from a monaural source. By storingmeasured HRTF (or HRIR) values for various source locations, thelocation of a source can be interactively varied simply by choosing andapplying the appropriate stored values to the sound source to produceleft and right channel outputs. A number of commercial 3D audio systemsexist utilizing this principle. Rather than storing values, the HRTF canbe modeled but this requires considerably more processing power.

The generation of binaural signals as described above is directlyapplicable to headphone systems. However, the situation is more complexwhere stereo loudspeakers are used for sound output because sound fromboth speakers can reach both ears. In one solution, the transferfunctions between each speaker and each ear are additionally derived andused to try to cancel out cross-talk from the left speaker to the rightear and from the right speaker to the left ear.

Other approaches to those outlined above for the generation of 3D audiofields are also possible as will be appreciated by persons skilled inthe art. Regardless of the method of generation of the audio field, most3D audio systems are, in practice, generally effective in achievingazimuth positioning but less effective for elevation and range. However,in many applications this is not a particular problem since azimuthpositioning is normally the most important. As a result, systems for thegeneration of audio fields giving the perception of physically separatedsound sources range from full 3D systems, through two dimensionalsystems (giving, for example, azimuth and elevation position variation),to one-dimensional systems typically giving only azimuth positionvariation (such as a standard stereo sound system). Clearly, 2D andparticularly ID systems are technically less complex than 3D systems asillustrated by the fact that stereo sound systems have been around forvery many years.

In terms of user experience, headphone-based systems are inherently“head stabilized”—that is, the generated audio field rotates with thehead and thus the position of each sound source appears stable withrespect to the user's head. In contrast, loudspeaker-based systems areinherently “world stabilized” with the generated audio field remainingfixed as the user rotates their head, each sound source appearing tokeep its absolute position when the hearer's head is turned. In fact, itis possible to make headphone-based systems “world stabilized” orloudspeaker-based systems “head stabilized” by using head-trackerapparatus to sense head rotation relative to a fixed frame of referenceand feed corresponding signals to the audio field generation system,these signals being used to modify the sound source positions to achievethe desired effect. A third type of stabilization is also sometimes usedin which the audio field rotates with the user's body rather than withtheir head so that a user can vary the perceived positions of the soundsources by rotating their head; such “body stabilized” systems can beachieved, for example, by using a loudspeaker-based system with smallloudspeakers mounted on the user's upper body or by a headphone—basedsystem used in conjunction with head tracker apparatus sensing headrotation relative to the user's body.

As regards the purpose of the generated audio field, this is frequentlyused to provide a complete user experience either alone or inconjunction with other artificially-generated sensory inputs. Forexample, the audio field may be associated with a computer game or otherartificial environment of varying degree of user immersion (includingtotal sensory immersion). As another example, the audio field may begenerated by an audio browser operative to represent page structure byspatial location.

Alternatively, the audio field may be used to supplement a user's realworld experience by providing sound cues and information relevant to theuser's current real-world situation. In this context, the audio field isproviding a level of “augmented reality”.

It is an object of the present invention to facilitate clearpresentation in audio user interfaces in which sound sources aresynthesised in an audio field.

SUMMARY OF THE INVENTION

According to one aspect of the present invention, there is provided anaudio user-interfacing method in which items are represented in an audiofield by corresponding synthesized sound sources from where soundsrelated to the items appear to emanate, the method comprising the stepsof:

-   (a) determining, for each said sound source, an associated rendering    position at which the sound source is normally to be synthesized to    sound in the audio field;-   (b) modifying said rendering positions of at least some of the sound    sources to dilate a region of the audio field set relative to a    focus reference; and-   (c) generating, using audio output devices, an audio field in which    said sound sources are synthesized at their associated rendering    positions after any modification effected by step(b).

According to another aspect of the present invention, there is providedan audio user-interfacing method in which items are represented in anaudio field by corresponding synthesized sound sources from where soundsrelated to the items appear to emanate, the method involving:

-   (a)—determining for each sound source a rendering position at which    the sound source is to be synthesized to sound in the audio field,    these positions being such that the audio field is dilated in a    region of the audio field set relative to a focus reference;-   (b)—generating an audio field in which said sound sources are    synthesized at their associated rendering positions;-   (c)—moving the focus reference relative to the audio field thereby    changing which sound sources are present in the region of the audio    field subject to dilation.

According to a further aspect of the present invention, there isprovided apparatus for providing an audio user interface in which itemsare represented in an audio field by corresponding synthesized soundsources from where sounds related to the items appear to emanate, theapparatus comprising:

-   -   rendering-position determining means for determining, for each        said sound source, an associated rendering position at which the        sound source is to be synthesized to sound in the audio field;    -   position-modifying means for modifying said rendering positions        of at least some of the sound sources to dilate a region of the        audio field set relative to a focus reference; and    -   rendering means, including audio output devices, for generating        an audio field in which said sound sources are synthesized at        their associated rendering positions, after any modification by        the position-modifying means, to provide sounds related to the        items concerned.

According to a still further aspect of the present invention, there isprovided apparatus for providing an audio user interface in which itemsare represented in an audio field by corresponding synthesized soundsources from where sounds related to the items appear to emanate, theapparatus comprising:

-   -   rendering-position determination functionality for determining        for each sound source a rendering position at which the sound        source is to be synthesized to sound in the audio field, these        positions being such that the audio field is dilated in a region        of the audio field set relative to a focus reference;    -   audio rendering functionality, including audio output devices,        generating an audio field in which said sound sources are        synthesized at their associated rendering positions; and    -   input functionality for moving the focus reference relative to        the audio field thereby changing which sound sources are present        in the region of the audio field subject to dilation.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the invention will now be described, by way ofnon-limiting example, with reference to the accompanying diagrammaticdrawings, in which:

FIG. 1 is a functional block diagram of a first audio-field generatingapparatus;

FIG. 2 is a diagram illustrating a coordinate system for positions in aspherical audio field;

FIG. 3 is a diagram illustrating rotation of an audio field relative toa presentation reference vector;

FIG. 4 is a diagram illustrating a user exploring a body-stabilizedaudio field by head rotation;

FIG. 5 is a diagram illustrating a user exploring a body-stabilizedaudio field by rotating the field in azimuth;

FIG. 6 is a diagram illustrating a general cylindrical organization ofan audio field;

FIG. 7 is a diagram illustrating a first specific form of the FIG. 6cylindrical organization;

FIG. 8 is a diagram illustrating a second specific form of the FIG. 6cylindrical organization;

FIG. 9 is a functional block diagram of a variant of the FIG. 1apparatus;

FIG. 10 is a functional block diagram of a second audio-field generatingapparatus;

FIG. 11 is a diagram illustrating the operation of a focus expander ofthe FIG. 10 apparatus to expand an audio field, the user facing in thesame direction as an audio field reference vector;

FIG. 12 is a further diagram illustrating the operation of the focusexpander, the user in this case facing in a different direction to theaudio field reference vector;

FIG. 13 is a diagram illustrating the operation of a segment mutingfilter of the FIG. 10 apparatus;

FIG. 14 is a diagram illustrating the operation of a cyclic mutingfilter of the FIG. 10 apparatus;

FIG. 15 is a diagram illustrating the operation of a collectioncollapser of the FIG. 10 apparatus;

FIG. 16 is a diagram illustrating the operation of a range sound setterof the FIG. 10 apparatus;

FIG. 17 is a diagram illustrating the concept of the range sound setterapplied to a context of a fixed device being approached by a person;

FIG. 18 is a functional block diagram showing further detail of the FIG.10 apparatus;

FIG. 19 is a diagram showing a relationship between loudness of a speechinput and a range gate set by the FIG. 10 apparatus for limiting thesearch space of a speech recognizer of the apparatus;

FIG. 20 is a diagram of a trackball type of input device usable by theFIG. 10 apparatus;

FIG. 21 is a diagram showing a trackball input device similar to FIG. 20but including a first form of visual orientation indicator arrangement;

FIG. 22 is a block diagram of functionality for determining theorientation of the audio field relative to an indicator reference;

FIG. 23 is a diagram showing a trackball input device similar to FIG. 20but including a second form of visual orientation indicator arrangement;and

FIG. 24 is a diagram of another form of input device usable by the FIG.10 apparatus, this device being suitable where the apparatus is arrangedto produce a cylindrical audio field; and

BEST MODE OF CARRYING OUT THE INVENTION

The forms of apparatus to be described below are operative to produce anaudio field to serve as an audio interface to services such ascommunication services (for example, e-mail, voice mail, fax, telephone,etc.), entertainment services (such as internet radio), informationresources (including databases, search engines and individualdocuments), transactional services (for example, retail and banking websites), augmented-reality services, etc.

When the apparatus is in a “desktop” mode, each service is representedin the audio field through a corresponding synthesized sound sourcepresenting an audio label (or “earcon”) for the service. The audio labelassociated with a service can be constituted by any convenient audioelement suitable for identifying that service—for example, an audiolabel can be the service name, a short verbal descriptor, acharacteristic sound or jingle, or even a low-level audio feed from theservice itself. The sound sources representing the services aresynthesized to sound, to a user, as though they exist at respectivelocations in the audio field using any appropriate spatialisationmethod; these sound sources do not individually exist as physical soundoutput devices though, of course, such devices are involved in theprocess of synthesizing the sound sources. Furthermore, the soundsources only have a real-world existence to the extent thatservice-related sounds are presented at the sound-source locations.Nevertheless, the concept of sound sources located at specific locationsin the audio field is useful as it enables the sound content that is tobe presented in respect of a service to be disassociated from thelocation and other presentation parameters for those sounds, theseparameters being treated as associated with the corresponding soundsource. Thus, the present specification is written in terms of suchsound sources spatialized to specific locations in the audio field.

Upon a service presented through a sound source being selected (in amanner to be described hereinafter), the apparatus changes from thedesktop mode to a service mode in which only the selected service isoutput, a full service audio feed now being presented in whatever soundspatialisation is appropriate for the service. When a user has finishedusing the selected service, the user can switch back to the desktopmode.

It will be appreciated that other possibilities exist as to how theservices are presented and accessed—for example, the feed from aselected service can be output simultaneously with backgroundpresentation of audio labels for the other available services.Furthermore, a service can provide its data in any form capable of beingconverted in audible form; for example, a service may provide its audiolabel in text form for conversion by a text-to-speech converter intoaudio signals, and its full service feed as digitised audio waveformsignals.

It is also possible in the desktop mode to use more than one soundsource to represent a particular service and/or to associate more thanone audio label with each sound source as will be seen hereinafter.

Audio Field Organisation—Spherical Field Example

Considering now the first apparatus (FIG. 1), in the form of theapparatus primarily to be described below, the audio field is a 2D audiofield configured as the surface of a sphere (or part of a sphere). Sucha spherical-surface audio field is depicted in FIG. 2 where aspatialised sound source 40 (that is, a service audio label that hasbeen generated so as to appear to come from a particular location in theaudio field) is represented as a hexagon positioned on the surface of asphere 41 (illustrated in dashed outline). It maybe noted that althoughsuch a spherical surface exists in three-dimensional space, the audiofield is considered to be a 2 dimensional field because the position ofspatialised sound sources in the audio field, such as source 40, can bespecified by two orthogonal measures; in the present case these measuresare an azimuth angle X° and an elevation angle Y°. The azimuth angle ismeasured relative to an audio-field reference vector 42 that lies in ahorizontal plane 43 and extends from the centre of sphere 41. Theelevation angle is the angle between the horizontal and the line joiningthe centre of the sphere and the sound source 40.

In fact, the FIG. 1 apparatus is readily adapted to generate a 3D audiofield with the third dimension being a range measure Z, also depicted inFIG. 2, that is the distance from the centre of sphere 41 to thespatialised sound source 40. Conversely, the FIG. 1 apparatus can beadapted to generate a 1D audio field by doing away with the elevationdimension of the spatialised sound sources.

The FIG. 1 apparatus supports azimuth rotation of the audio field, thispotentially being required for implementing a particular stabilization(that is, for example, head, body, vehicle or world stabilization) ofthe audio field as well as providing a way for the user to explore theaudio field by commanding a particular rotation of the audio field. Asis illustrated in FIG. 3, the azimuth rotation of the field can beexpressed in terms of the angle R between the audio-field referencevector 42 and a presentation reference vector 44. This presentationreference vector corresponds to the straight-ahead centreline directionfor the configuration of audio output devices 11 being used. Thus, for apair of fixed, spaced loudspeakers, the presentation reference vector 44is the line of equidistance from both speakers and is therefore itselffixed relative to the world; for a set of headphones, the presentationreference vector 44 is the forward facing direction of the user andtherefore changes its direction as the user turns their head. When thefield rotation angle R=0°, the audio-field reference vector 42 isaligned with the presentation reference vector 44. The user is at leastnotionally located at the origin of the presentation reference vector.

The actual position at which a service-representing sound source is tobe rendered in the audio output field (its “rendering position”) by theFIG. 1 apparatus, must be derived relative to the presentation referencevector since this is the reference used by the spatialisation processor10 of the apparatus. The rendering position of a sound source is acombination of the intended position of the source in the audio fieldjudged relative to the audio-field reference vector, and the currentrotation of the audio field reference vector relative to thepresentation reference vector.

As already intimated, apart from any specific azimuth rotation of theaudio field deliberately set by the user, the audio field may need to berotated in azimuth to provide a particular audio-field stabilisation.Whether this is required depends on the selected audio-fieldstabilization and the form of audio output devices. Thus, by way ofexample, unless otherwise stated, it will be assumed below that theaudio output devices 11 of FIG. 1 apparatus are headphones and the audiofield is to be body-stabilised so that the orientation of the audiofield relative to the user's body is unaltered when the user turns theirhead—this is achieved by rotation of the audio field relative to thepresentation reference vector for which purpose a suitable head-trackersensor 33 is provided to measure the azimuth rotation of the user's headrelative to its straight-ahead position (that is, relative to the user'sbody). As the user turns their head, the angle measured by sensor 33 isused to rotate the audio field by the same amount but in the oppositedirection thereby stabilising the rendering positions of the soundsources relative to the user's body.

It will be appreciated that had it been decided to head-stabilise thefield, then for audio output devices in the form of headphones, it wouldhave been unnecessary to modify the orientation of the audio field asthe user turned their head and, in this case, there would be no need forthe head-tracker sensor 33. This would also be true had the audio outputdevices 11 taken the form of fixed loudspeakers and the audio field wasto be world-stabilized. Where headphones are to be used and the audiofield is to be world stabilised, the orientation of the audio field mustbe modified by any change in orientation of the user's head relative tothe world, whether caused by the user turning their head or by bodymovements; a suitable head-tracker can be provided by a head-mountedelectronic compass. Similarly, if the audio output devices 11 are to beprovided by a vehicle sound system and the audio field is to be worldstabilised, the orientation of the audio field must be modified by anychange in orientation of the vehicle as determined by any suitablesensor. It may be generally be noted that where a user is travelling ina vehicle, the latter serves as a local world so that providing vehiclestabilisation of the audio field is akin to providing worldstabilisation (whether the audio output devices are headphones, bodymounted or vehicle mounted) but with any required sensing of userhead/body rotation relative to the world now being done with respect tothe vehicle.

It is also to be noted that the audio-field rotation discussed aboveonly concerned azimuth rotation—that is, rotation about a vertical axis.It is, of course, also possible to treat rotation of the field inelevation in a similar manner both to track head movements (nodding upand down) to achieve a selected stabilisation and to enable the user tocommand audio-field elevation-angle changes; appropriate modificationsto the FIG. 1 apparatus to handle rotation in elevation in this way willbe apparent to persons skilled in the art.

Considering FIG. 1 in more detail, services are selected by subsystem13, these services being either local (for example, an applicationrunning on a local processor) or accessible via a communications link 20(such as a radio link or fixed wire connection providing internet orintranet access). The services can conveniently be categorised intogeneral services such as e-mail, and services that have relevance to theimmediate vicinity (augmentation services). The services are selected byselection control block 17 according to predetermined user-specifiedcriteria and possibly also by real-time user input provided via anysuitable means such as a keypad, voice input unit or interactivedisplay.

A memory 14 is used to store data about the selected services with eachsuch service being given a respective service ID. For each selectedservice, memory 14 holds access data (e.g. address of service executableor starting URL) and data on the or each sound source specified by theservice or user to be used to represent the service with each such soundsource being distinguished by a suitable suffix to the service ID. Foreach sound source, the memory holds data on the or each associated audiolabel, each label being identified by a further suffix to the suffixedservice ID used to identify the sound source. The audio labels for theselected services are either provided by the services themselves to thesubsystem 13 or are specified by the user for particular identifiedservices. The labels are preferably provided and stored in text-form forconversion to audio by a text-to-speech converter (not shown) as andwhen required by the spatialisation processor. Where the audio labelassociated with a service is to be a low-level live feed, memory 14holds an indicator indicating this. Provision may also be made fortemporarily replacing the normal audio label of a service sound sourcewith a notification of a significant service-related event (for example,where the service is an e-mail service, notification of receipt of amessage may temporarily substitute for the normal audio label of theservice).

As regards the full service feed of any particular service, this is notoutput from subsystem 13 until that service is chosen by the user byinput to output selection block 12.

Rather than the services to be represented in the audio interface beingselected by block 17 from those currently found to be available, a setof services to be presented can be pre-specified and the relatedsound-source data (including audio labels) for these services stored inmemory 14 along with service identification and access data. In thiscase, when the apparatus is in its “desktop” mode, the services in thepre-specified set of services are represented in the output audio fieldby the stored audio labels without any need to first contact theservices concerned; upon a user selecting a service and the apparatuschanging to its service mode, the service access data for the selectedservice is used to contact that service for a full service feed.

With respect to the positioning of the service-representing soundsources in the audio field when the apparatus is in its desktop mode,each service may provide position information either indicating asuggested spatialised position in the audio field for the soundsource(s) through which the service is to be represented, or giving areal-world location associated with the service (this may well be thecase in respect of an augmented reality service associated with alocation in the vicinity of the user). Where a set of services ispre-specified, then this position information can be stored in memory 14along with the audio labels for the services concerned.

For each service-representing sound source, it is necessary to determineits final rendering position in the output audio field taking account ofa number of factors. This is done by injecting a sound-source data iteminto a processing path involving elements 21 to 30. This sound-sourcedata item comprises a sound source ID (such as the related suffixedservice ID) for the sound source concerned, any service-suppliedposition information for the sound source, and possibly also the servicetype (general service/augmentation service). The subsystem 13 passeseach sound-source data item to a source-position set/modify block 23where the position of the sound source is decided relative to theaudio-field reference vector, either automatically on the basis of thesupplied type and/or position information, or from user input 24provided through any suitable input device including a keypad, keyboard,voice recognition unit, or interactive display. These positions areconstrained to conform to the desired form (spherical or part spherical;1D, 2D, or 3D) of the audio field. The decided position for each sourceis then temporarily stored in memory 25 against the source ID.

Provision of a user input device for modifying the position of eachsound source relative to the audio field reference, enables the user tomodify the layout of the service-representing sound sources (that is,the dispositions of these sound sources relative to each other) asdesired.

With respect to a service having an associated real-world location(typically, an augmented reality service), whilst it is possible toposition the corresponding sound source in the audio field independentlyof the relationship between the associated real-world location of theservice and the location of the user, it will often be desired to placethe sound source in the field at a position determined by the associatedreal-world location and, in particular, in a position such that it liesin the same direction relative to the user as the associated real-worldlocation. In this latter case, the audio field will generally beworld-stabilised to maintain the directional validity of the soundsource in the audio field presented to the user; for the same reason,user-commanded rotation of the audio field should be avoided orinhibited. Positioning a sound source according to an associatedreal-world location is achieved in the present apparatus by a real-worldlocation processing functional block 21 that forms part of thesource-position set/modify block 23. The real-world location processingfunctional block 21 is arranged to receive and store real-worldlocations passed to it from subsystem 13, these locations being storedagainst the corresponding source IDs. Block 21 is also supplied on input22 with the current location of the user determined by any suitablemeans such as a GPS system carried by the user, or nearby locationbeacons (such as maybe provided at point-of-sale locations). The block21 first determines whether the real-world location associated with aservice is close enough to the user to qualify the corresponding soundsource for inclusion in the audio field; if this test is passed, theazimuth and elevation coordinates of the sound source are set to placethe sound source in the audio field in a direction as perceived by theuser corresponding to the direction of the real world location from theuser. This requires knowledge of the real-world direction of pointing ofthe un-rotated audio-field reference vector 42 (which, as noted above,is also the direction of pointing of the presentation reference vector).This can be derived for example, by providing a small electronic compasson a structure carrying the audio output devices 11, since this enablesthe real-world direction of pointing of presentation reference vector 44to be measured; by noting the rotation angle of the audio-fieldreference vector 42 at the moment the real-world direction of pointingof vector 44 is measured, it is then possible to derive the real-worlddirection of pointing of the audio-field reference vector 42 (assumingthat the audio field is being world-stabilised). It maybe noted that notonly will there normally be a structure carrying the audio outputdevices 11 when these are constituted by headphones, but this is alsothe case in any mobile situation (for example, in a vehicle) whereloudspeakers are involved.

If the audio field is a 3D field, then as well as setting the azimuthand elevation coordinates of the sound source to position it in the samedirection as the associated real-world location, block 21 also sets arange coordinate value to represent the real world distance between theuser and the real-world location associated with the sound source.

Of course, as the user moves in space, the block 21 must reprocess itsstored real-world location information to update the position of thecorresponding sound sources in the audio field. Similarly, if updatedreal-world location information is received from a service, then thepositioning of the sound source in the audio field must also be updated.

Returning to a general consideration of the FIG. 1 apparatus, anaudio-field orientation modify block 26 is used to specify any requiredchanges in orientation (angular offset) of the audio-field referencevector relative to presentation reference vector. In the present examplewhere the audio field is to be body-stabilized and the output audiodevices are headphones, the apparatus includes the afore-mentioned headtracker sensor 33 and this sensor is arranged to provide a measure ofthe turning of a user's head relative to their body to a first input 27of the block 26. This measure is combined with any user-commanded fieldrotation supplied to a second input of block 26 in order to derive afield orientation angle that is stored in memory 29.

As already noted, where headphones are used and the audio field is to beworld stabilised (for example, where augmented-reality service soundsources are to be maintained in positions in the field consistent withtheir real world positions relative to the user), then the head-trackersensor needs to detect any change in orientation of the user's headrelative to the real world so that the audio field can be given acounter rotation. Where the user is travelling in a vehicle and theaudio field is to be vehicle-stabilised, the rotation of the user's headis measured relative to the vehicle (the user's “local” world, asalready noted).

Each source position stored in memory 25 is combined by combiner 30 withthe field orientation (rotation) angle stored in memory 29 to derive arendering position for the sound source, this rendering position beingstored, along with the source ID, in memory 15. The combiner operatescontinuously and cyclically to refresh the rendering positions in memory15.

Output selection block 12 sets the current apparatus mode according touser input, the available modes being a desktop mode and a service modeas already discussed above. When the desktop mode is set, thespatialisation processor 10 accesses the rendering position memory 15and the memory 14 holding the service audio labels to generate an audiofield, via audio output devices 11, in which the (or thecurrently-specified) audio label associated with each sound source isspatialized to a position set by the corresponding rendering position inmemory 15. In generating the audio-label field, the processor 10 canfunction asynchronously with respect to the combiner 30 due to theprovision of memory 15. The spatialisation processor 10 operatesaccording to any appropriate sound spatialisation method, includingthose mentioned in the introduction to the present specification. Thespatialisation processor 10 and audio output devices together form arendering subsystem serving to render each sound source at its derivedfinal rendering position.

When the service mode is set, the full service audio feed for the chosenservice is rendered by the spatialisation processor 10 according towhatever position information is provided by the service. It will beappreciated that, although not depicted, this service positioninformation can be combined with the field orientation angle informationstored in memory 29 to achieve the same stabilization as for theaudio-field containing the service audio labels; however, this is notessential and, indeed, the inherent stabilization of the audio outputdevices (head-stabilised in the case of headphones) maybe moreappropriate for the full service mode.

As an alternative to the full service feed being spatialised by thespatialisation processor 10, the full service feed may be provided aspre-spatialized audio signals and fed directly to the audio outputdevices.

With the FIG. 1 apparatus set to provide a body-stabilised audio fieldthrough headphones, the user can explore the audio field in two ways,namely by turning their head and by rotating the audio field. FIG. 4illustrates a user turning their head to explore a 2D audio fieldrestricted to occupy part only of a spherical surface. In this case, sixspatialised sound sources 40 are depicted. Of these sources, one source40A is positioned in the audio field at an azimuth angle of X₁° andelevation angle Y₁° relative to the audio-field reference vector 42. Theuser has not commanded any explicit rotation of the audio field.However, the user has turned their head through an angle X₂° towards thesource 40A. In order to maintain body-stabilisation of the audio field,the audio-field reference vector 42 has been automatically rotated anangle (−X₂°) relative to the presentation reference vector 44 to bringthe vector 42 back in line with the user's body straight aheaddirection; the rendering position of the source relative to thepresentation reference vector is therefore:

-   -   Azimuth=X₁−X₂°    -   Elevation=Y₁°        this being the position output by combiner 30 and stored in        memory 15. The result is that turning of the user's head does        indeed have the effect of turning towards the sound source 40A.

FIG. 5 illustrates, for the same audio field as represented in FIG. 4,how the user can bring the sound source 40A to a position directly aheadof the user by commanding a rotation of (−X₁°) of the audio field byuser input 28 to block 26 (effected, for example, by a rotary inputdevice). The azimuth rendering position of the sound source 40A becomes(X₁°−X₁°), that is, 0°—the source 40A is therefore rendered in line withthe presentation reference vector 44. Of course, if the user turns theirhead, the source 40A will cease to be directly in front of the useruntil the user faces ahead again.

Audio Field Organisation—Cylindrical Field Example

The FIG. 1 apparatus can be adapted to spatialize the sound sources 40in an audio field conforming to the surface of a vertically-orientatedcylinder (or part thereof). FIG. 6 depicts a general case where theaudio field conforms to a notional cylindrical surface 50. Thiscylindrical audio field, like the spherical audio field previouslydescribed with reference to FIG. 2, is two dimensional inasmuch as theposition of a sound source 40 in the field it can be specified by twocoordinates, namely an azimuth angle X° and an elevation (height)distance Y, both measured relative to an horizontal audio-fieldreference vector 52. It will be appreciated that a 3D audio field can bespecified by adding a range coordinate Z, this being the distance fromthe axis of the cylindrical audio field. As with the spherical audiofield described above, the cylindrical audio field may be rotated(angularly offset by angle R°) relative to a presentation referencevector 54, this being done either in response to a direct user commandor to achieve a particular field stabilisation in the same manner asalready described above for the spherical audio field. In addition, theaudio field can be axially displaced to change the height (axial offset)of the audio-field reference vector 52 relative to the presentationreference vector 54.

Since it is possible to accommodate any desired number of sound sourcesin the audio field without over crowding simply by extending theelevation axis, there is a real risk of a “Tower of Babel” being createdif all sound sources are active together. Accordingly, the general modelof FIG. 6 employs a concept of a focus zone 55 which is a zone of thecylindrical audio field bounded by upper and lower elevation valuesdetermined by a currently commanded height H so as to keep the focuszone fixed relative to the assumed user position (the origin of thepresentation reference vector); within the focus zone, the sound sources40 are active, whilst outside the zone the sources 40 are muted(depicted by dashing of the hexagon outline of these sources in FIG. 6)except for a limited audio leakage 53. In FIG. 6, the focus zone (whichis hatched) extends by an amount C above and below the commanded heightH (and thus has upper and lower elevation values of (H+C) and (H−C)respectively. In the illustrated example, H=0 and C is a constant; Cneed not be constant and it would be possible, for example, to make itsvalue dependent on the value of the commanded height H.

The general form of cylindrical audio field shown in FIG. 6 can beimplemented in a variety of ways with respect to how leakage into thefocus zone is effected and how a user moves up and down the cylindricalfield (that is, changes the commanded height and thus the current focuszone). FIGS. 7 and 8 illustrate two possible implementations in the casewhere the audio field is of semi-cylindrical form (azimuth range from+90° to −90°).

In FIG. 7, leakage takes the form of the low-volume presence of soundsources 40W in upper and lower “whisper” zones 56, 57 positionedadjacent the focus zone 55. Also, the commanded height value iscontinuously variable (as opposed to being variable in steps). Theresult is that the user can effectively slide up and down the cylinderand hear both the sound sources 40 in the focus zone and, at a lowervolume, sound sources 40W in the whisper zones.

In FIG. 8, the service sound sources are organised to lie at a number ofdiscrete heights, in this case, four possible heights effectivelycorresponding to four “floors” here labelled “1” to “4”. Preferably,each “floor” contains sound sources associated with services all of thesame type with different floors being associated with different servicetypes. The user can only command step changes in height corresponding tomoving from floor to floor (the extent of the focus zone encompassingone floor). Leakage takes the form of an upper and lower advisory soundsource 60, 61 respectively positioned just above and just below thefocus zone at an azimuth angle of 0°. Each of these advisory soundsources 60, 61 provides a summary of the services (for example, in termsof service types) available respectively above and below the currentfocus zone. This permits a user to determine whether they need to go upor down to find a desired service.

It will be appreciated that the forms of leakage used in FIGS. 7 and 8can be interchanged or combined and that the FIG. 8 embodiment canprovide for sound sources 40 on the same floor to reside at differentheights on that floor. It is also possible to provide each floor of theFIG. 8 embodiment with a characteristic audio theme which rather thanbeing associated with a particular source (which is, of course,possible) is arranged to surround the user with no directionality; byway of example, a floor containing museum services could have aclassical music theme.

In arranging for the FIG. 1 apparatus to implement a cylindrical audiofield such as depicted in any of FIGS. 4-6, the positions set for thesound sources by block 23 are specified in terms of the describedcylindrical coordinate system and are chosen to conform to a cylindricalor part-cylindrical organisation in 1, 2, or 3D as required. Theorientation and vertical positioning of the audio field reference vector42 are set by block 26, also in terms of the cylindrical coordinatesystem. Similarly, combiner 30 is arranged to generate the sound-sourcerendering positions in terms of cylindrical coordinates. Thespatialisation processor must therefore either be arranged to understandthis coordinate system or the rendering positions must be converted to acoordinate system understood by the spatialisation processor 10 beforethey are passed to the processor. This latter approach is preferred andthus, in the present case, assuming that the spatialisation processor isarranged to operate in terms of the spherical coordinate systemillustrated in FIG. 2, a converter 66 (see FIG. 9) is provided upstreamof memory 15 to convert the rendering positions from cylindricalcoordinates to spherical coordinates.

Whilst it would be possible to use a single coordinate system throughoutthe apparatus regardless of the form of audio field to be produced (forexample, the positions of the sound sources in the cylindrical audiofield could be specified in spherical coordinates), this complicates theprocessing because with an appropriately chosen coordinate system mostoperations are simple additions or subtractions applied independently tothe individual coordinates values of the sound sources; in contrast, if,for example, a spherical coordinate system is used to specify thepositions in a cylindrical field, then commanded changes in the fieldheight (discussed further below) can no longer simply beadded/subtracted to the sound source positions to derive their renderingheights but instead involve more complex processing affecting bothelevation angle and range. Indeed, by appropriate choice of coordinatesystem for different forms of audio field, equivalent operations withrespect to the fields translate to the same operations (generallyadd/subtract) on the coordinate values being used so that the operationof the elements 25, 26, 29 and 30 of the apparatus is unchanged. In thiscase, adapting the apparatus to a change in audio-field form, simplerequires the block 23 to use an appropriate coordinate system and forconverter 66 to be set to convert from that coordinate system to thatused by the spatialisation processor 10.

With respect to adaptation of the FIG. 1 apparatus to provide therequired capability of commanding changes in height for the cylindricalaudio field systems illustrated in FIGS. 4-6, such height changescorrespond to the commanding of changes in the elevation angle alreadydescribed for the case of a spherical audio field. Thus, a height changecommand is supplied to the block 26 to set a field height value (anaxial offset between the field reference vector and the presentationreference vector) which is then combined with the elevation distancevalue Y of each sound source to derive the elevation value for therendering position of the source.

As regards how the focus zone and leakage features are implemented, FIG.9 depicts a suitable variation of the FIG. 1 apparatus for providingthese features. In particular, a source parameter set/modify block 70 isinterposed between the output of combiner 30 and the converter 66. Thisblock 70 comprises one or more units for setting and/or modifying one ormore parameters associated with each sound source to condition how thesound source is to be presented in the audio field. As will be seenhereinafter with respect to the FIG. 10 apparatus, the block 70 caninclude a range of different type of units that may modify the renderingposition of a source and/or set various sounding effect parameters forthe source. In the present case, the block 70 comprises a cylindricalfilter 71 that sets a audibility (volume level) sounding-effectparameter for each sound source. The set parameter value is passed tomemory 15 for storage along with the source ID and rendering position.When the spatialisation processor comes to render the sound source audiolabel according to the position and audibility parameter value stored inmemory 15, it passes the audibility value to a sounding effector 74 thatconditions the audio label appropriately (in this case, sets its volumelevel).

In the case of the FIG. 7 arrangement, the cylinder filter 71 isresponsive to the current field height value (as supplied from memory 29to a reference input 72 of block 70) to set the audibility parametervalue of each sound source: to 100% (no volume level reduction) forsound sources in the focus zone 55; to 50% for sound sources in the“whisper” zones 56 and 57; and to 0% (zero volume) for all other soundsources. As a result, the sounding effector 74 mutes out all soundsources not in the focus or whisper zones, and reduces the volume levelof sound sources in the whisper zones.

In the case of the FIG. 8 arrangement, the cylinder filter 71 performs asimilar function except that now there are no whisper zones. As regardsthe upper and lower advisory sound sources 60 and 61, the subsystem 13effectively creates these sources by:

-   -   creating a ghost advisory service in memory 14 with two sound        sources, the IDs of these sources being passed to block 23 as        for any other service;    -   creating for each sound source a respective set of summary audio        labels, each set being stored in memory 14 and specifying for        each floor an appropriate label summarising the service types        either above or below the current floor, depending on the set        concerned.

The source IDs passed to the block 23 are there associated with nullposition data before being passed on via memory 25 and combiner 30 toarrive at the cylinder filter 71 of block 70. The filter 71 recognisesthe source IDs as upper and lower advisory sound source IDs andappropriately sets position data for them as well as setting theaudibility parameter to 100% and setting a parameter specifying whichsummary audio label is appropriate for the current floor. This enablesthe spatialisation processor to retrieve the appropriate audio labelwhen it comes to render the upper or lower advisory sound source.

It will be appreciated that partially or fully muting sound sourcesoutside of a focus zone can also be done where the apparatus is set togenerate a spherical audio field. In this case, the apparatus includesblocks 70 and 74 but now the cylinder filter 71 is replaced by a“spherical filter” muting out all sound sources beyond a specifiedangular distance from a current facing direction of the user. Thecurrent facing direction relative to the presentation reference vectoris derived by block 26 and supplied to the filter 71. It maybe notedthat in the case where the audio output devices 11 are constituted byheadphones, the direction of facing of the user corresponds to thepresentation reference vector so it is a simple matter to determinewhich sound sources have rendering positions that are more than a givenangular displacement from the facing direction. Along with theimplementation of a focus zone for a spherical audio field, it is, ofcourse, also possible to provide the described implementations of aleakage feature.

Multiple Audio Sub-Fields

FIG. 10 shows a second apparatus for producing an audio field to serveas an audio interface to services. This apparatus is similar to the FIG.9 variant of the first apparatus but provides for multiple audio“sub-fields” and has a variety of sound-source parameter conditioningunits for facilitating a clear audio presentation. Elements of the firstand second apparatus that have similar functionality have been given thesame reference numerals and their description will not be repeated belowfor the second apparatus except where there is modification offunctionality to accommodate features of the second apparatus.

The second apparatus, like the first apparatus, is capable of producing(part) spherical or part (cylindrical) 1D, 2D or 3D audio fields (or,indeed, any other form of audio field) according to the positions setfor the sound sources by block 23.

As mentioned, the FIG. 10 apparatus provides for multiple “sub-fields”.Each sub-field may be considered as an independent audio field that canbe rotated (and, in the case of a cylindrical field, verticallyre-positioned) by changing the offset between the presentation referencevector and an audio-field reference specific to the sub-field. Further,each sub-field can have a different stabilization set for it—thus, forexample, sound sources representing general services can be assigned toa head-stabilised sub-field whilst sound sources representingaugmented-reality services can be assigned to a world-stabilisedsub-field. The rotation/displacement of each sub-field and the settingof its stabilization is done by block 26 with the resultant values beingstored in memory 29. Whether or not the block 26 modifies theazimuth-angle value of a sub-field to reflect a sensed rotation of theuser's head will thus depend on the stabilization set for the sub-fieldand, as already described, on whether the audio output devices arehead-mounted, body-mounted, vehicle-mounted or fixed with respect to theworld (or, in other words, whether the presentation reference vector ishead, body, vehicle or world stabilised). To add flexibility to the FIG.10 apparatus, the current stabilisation of the presentation referencevector is fed to the block (see arrow) to enable the latter to make anyappropriate changes to the sub-field orientations as the user turns(and/or nods) their head.

Each service sound source is assigned by block 23 to a particularsub-field and an identifier of its assigned sub-field is stored with thesource ID in memory 25 along with the position of the sound sourcerelative to the audio-field reference associated with the assignedsub-field. The combiner 30 is supplied from memory 29 with therotation/displacement values of each sub-field and for each servicesound source combines the values of the related sub-field with thesound-source coordinate values; as a result, each sound source isimparted the rotations/displacements experienced by its sub-field. Foreach service sound source, the output of the combiner comprises sourceID, position data, and sub-field identifier.

As will be seen below, assigning sound sources to different sub-fieldsmay be done for reasons other than giving them different stabilizations;for example, it may be done to identify a group of service sound sourcesthat are to be subject to a particular source-parameter modificationprocess in block 70.

It should also be noted that different sub-fields may have differentdimensions and even different forms so that one sub-field could be a 2Dspherical surface whilst another sub-field could be of 3D cylindricalform.

Facilitating Clear Presentation

As well as the cylindrical filter 71, the source parameter set/modifyblock 70 includes a number of sound-source parameter conditioning units80 to 85 for facilitating a clear audio presentation. The function ofeach of these units will be described more fully below. It is to beunderstood that the units need not all be present or operationaltogether and various combinations of one or more units beingconcurrently active are possible; however, not all combinations areappropriate but this is a matter easilyjudged and will not beexhaustively detailed below. Also, certain units may need to effecttheir processing before others (for example, units that affect the finalrendering position of a sound source need to effect their processingbefore units that set sounding effect parameters in dependence on thefinal rendering position of a sound source); again, it will generally beapparent when such ordering issues are present and what ordering of theunits is required to resolve such issues and an exhaustive treatment ofthese matters will not be given below.

Unit 80 is a focus expander that serves to modify the renderingpositions of the sound sources to spread out the sound sources (that is,expand or dilate the audio field) in azimuth in the region of thecurrent direction of facing of the user (or other appropriate direction)in order to facilitate discrimination between sound sources. Referringto FIG. 11, this shows a field of 180° extent in azimuth with the usercurrently facing in the direction of the audio-field reference vector90. The focus expander 80 operates to linearly expand the 15° segments92 on both sides of the facing direction 91 into respective 45° segments93 (see the hatched zones). The remaining segments are correspondinglycompressed to maintain an overall 180° azimuth range—in this case, thisresults in two 75° segments 94 being compressed into respective 45°segments 95; as an alternative (not illustrated), the remaining segmentscould simply be angularly displaced from their normal positions withoutcompressing them.

For sub-fields that are head-stabilised, turning of the user's head doesnot change the 15° segments subject to expansion; however, azimuthrotation of such a sub-field does result in the expansion being appliedto different segments of the sub-field.

For sub-fields that are not head-stabilised, as the user turns theirhead, the segments subject to expansion change. This is illustrated inFIG. 12 where a user has turned to the right 75° relative to theaudio-field reference vector of a body-stabilised audio sub-field withan initial ±90° range either side of the reference vector. This resultsin the most clockwise 30° of the original field (segments 92) beingexpanded (symmetrically with respect to the facing direction) so thatnow the audio sub-field extends round further in the clockwise directionthan before. The remaining 150° segment 97 of the original audiosub-field is expanded into a 90° segment 98.

In order for the focus expander 80 to effect the required processing ofthe azimuth rendering positions of the sound sources, it is supplied(input 78 to block 70) with the angle of the facing direction relativeto the current presentation reference vector, this angle beingdetermined by the block 26 in dependence on the current stabilization ofthe presentation reference vector and the sensed head rotation. Ofcourse, where the presentation reference vector is head-stabilized (i.e.headphones are being used), the angle between the facing direction andthe presentation reference vector will be zero; in other cases it willgenerally correspond to the angle measured by the head-tracker sensor33. Given the facing direction angle relative to the presentationreference vector, and bearing in mind that the sound-source positionssupplied to block 70 are relative to that vector, it is astraightforward matter for the focus expander 80 to determine whichsound sources lie within the segments 92 and then make the requiredchanges to the azimuth values of the sound-source rendering positions ofthese sources in order to achieve the desired audio-field dilation;similarly, the rendering positions of the other sound sources areadjusted as required.

It will be appreciated that the user can be enabled to turn the focusexpander 80 on and off as desired. It is also possible to arrange forthe focus expander to be applied only to one or more selected sub-fieldsrather than to all fields indiscriminately. Furthermore, whilst thefocus expander has been described above as operating on azimuth angles,it could additionally or alternatively be caused to act on the elevationcoordinate values (whether angles or distances) and/or rangecoordinates. Again, whilst the expansion has been described above asbeing uniform (linear), it could be applied in a non-linear manner suchthat a larger expansion is applied adjacent the facing direction thanfurther away. The angle of application of the expansion effect can alsobe made adjustable.

Rather than the focus expander 80 expanding a region of the audio fieldset relative to the current facing direction, the focus expander can bearranged to expand a region set relative to some other direction (the‘focus reference direction’), such as a specific world-stabiliseddirection or the presentation reference vector. In this case, the focusexpander is provided with appropriate information from block 26 toenable it to determine the relative offset between the focus referencedirection and the presentation reference vector (this offset being, ofcourse, zero if the focus reference direction is set to be thepresentation reference vector).

Arrow 79 in FIG. 10 generally represents user input to block 70 whetherfor controlling the focus expander 80 or any other of the units of theblock. How the user input is derived is an implementation detail andmay, for example, be done by selection buttons, a graphical userinterface, or voice command input subsystem.

Unit 81 of the source-parameter set/modify block 70 is a segment mutingfilter 81 that is operative to change the audibility state of soundsources in user-specified segments of one, some or all the audiosub-fields (a default of all sub-fields is preferably set in the filter81 with the possibility of the user changing this default). Inparticular, the segment muting filter changes the audibility state ofsegment sound sources (in either direction) between un-muted and atleast partially muted by appropriately setting the value of anaudibility (sound volume) parameter of the sound sources. FIG. 13illustrates the effect of the segment muting filter in respect of anaudio sub-field, of 180° azimuth extent, shown developed into arectangular form 100 and with spatialised sound sources 40. In thisexample, the audio field is divided into five segments relative to theaudio-field reference vector, namely:

-   -   an “ahead” segment 101 extending in azimuth from +30° to −30°;    -   a “left” segment 102 extending in azimuth from −30° to −60°;    -   a “far left” segment 103 extending in azimuth from −60° to −90°;    -   a “right” segment 104 extending in azimuth from +30° to +60°;    -   a “far right” segment 105 extending in azimuth from +60° to        +90°.

The filter 81 acts to change the audibility parameter of each soundsource in a segment back and forth between 100% and 0% (or a preset lowlevel) in response to user input. Preferably, speech form input ispossible so that to mute sound sources in segment 102, the user needonly say “Mute Left” (FIG. 13 depicts these sounds sources as muted byshowing them in dashed outline). To bring back these sound sources tofull volume, the user says “Un-Mute Left”. As already described withrespect to the cylindrical filter 71, the sound volume specified by theaudibility parameter is implemented by sounding effector 74, theeffector being passed the parameter when the spatialisation processor 10requests to be supplied with the sound label for the sound sourceconcerned.

Preferably, the segments can be muted and un-muted independently of eachother. An alternative is to arrange for only one segment to be muted ata time with the selection for muting of a segment automaticallyun-muting any previously muted segment; the opposite is also possiblewith only one segment being un-muted at a time, the un-muting of asegment causing any previously un-muted segment to be muted. It is alsopossible to arrange for several segments to be muted simultaneously inresponse to a single command—for example, both the “left” and “far left”segments 102, 103 in FIG. 13 could be arranged to be muted in responseto a user command of “Mute All Left”.

The segments are pre-specified in terms of their azimuth angular extentrelative to the audio-field reference vectors by segmentation datastored in the segment muting filter or elsewhere. In order for thesegment muting filter to mute the sound sources corresponding to asegment to be muted, the filter needs to know the current azimuth anglebetween the audio field reference vectors and the presentation referencevector since the sound-source azimuth angles provided to the filter arerelative to the latter vector. The required angles between theaudio-field and presentation reference vectors is supplied on input 76from block 26 to block 70.

As an alternative to the segments being specified relative to theaudio-field reference vectors, the segments can be specified relative tothe facing direction of the user (which may, in fact, be more natural).In this case, the segment muting filter needs to know the angle betweenthe current facing direction and the presentation reference vector; asalready described, this angle is provided on input 78 to block 70. Afurther alternative is to pre-specify the segments relative to thepresentation reference vector (which, of course, for headphones is thesame as specifying the segments relative to the user's facingdirection).

Whilst segment muting has been described using segmentation in azimuth,it will be appreciated that the segmentation can be effected in anyappropriate manner (for example, in azimuth and elevation incombination) and the term ‘segment’ is herein used without anyconnotation regarding the form or shape encompassed.

Rather than a segment remaining muted until commanded to return to itsun-muted state, a muted segment can be arranged only to stay muted for alimited period and then to automatically revert to being un-muted.

Unit 82 is a cyclic muting filter. As depicted in FIG. 14 (which usesthe same field development as FIG. 13), this filter 82 works on thebasis that the sound sources 40 are divided into groups 110 to 114 andthe filter 82 operates cyclically to change the audibility state of thesound sources so as to at least partially mute out all but one group ofsources in turn—in FIG. 14, all groups except group 111 are currentlymuted. The un-muted group remains un-muted, for example, for 10 secondsbefore being muted (partially or fully) again. As with the segmentmuting filter, the filter 82 operates by setting the value of anaudibility parameter of each sound source. Rather than requiring a groupID to be assigned to each sound source and transferred along with thesound-source ID, position data, and sub-field identifier to the block70, grouping can be achieved by assigning a separate sub-field for eachgroup.

The grouping of sound sources can be effected automatically by servicetype (or more generally, one or more characteristics associated with theitem represented by the sound source concerned). Alternatively, thegrouping of the sound sources can be effected automatically according totheir positions in the audio field (possibly taking account theirrelation to the presentation reference vector, the audio field referencevectors, or user direction of facing). A further possibility is for thegrouping to be user specified (via block 23). In one possible groupingarrangement, each sound source is assigned to a respective groupresulting in each sound source being un-muted in turn. Preferably, theuser can also specify that one or more groups are not subject to cyclicmuting. Additionally, the user can be given the option of setting theun-muted duration for each group.

As already indicated, muted groups need not be fully muted. Where thesound sources are assigned to groups according to their positions, apossible muting pattern would be to fully mute sound sources in groupslying either side of the currently un-muted group of sources, and topartially mute the sound sources of all other groups.

Rather than the un-muting and muting of the groups being effected in anabrupt manner, the group whose limited period of being un-muted isending can be cross-faded with the group whose period of being un-mutedis next to occur.

Unit 83 is a collection collapser the basic purpose of which is torespond to a predetermined user command to collapse all sound sourcesthat are members of a specified collection of sound sources to a singlecollection-representing sound source at a particular location (which canbe head, body, vehicle or world stabilised). The member sound sources ofthe collection can be identified by a specific tag associated with eachsound source ID; however, it is convenient to assign all sound sourcesto be collapsed to the same sub-field and simply rely on the sub-fieldID to identify these sources to the block 70.

FIG. 15 illustrates the general effect of the collection collapser 83for a situation where all augmented-reality sound sources 40[AR] aremembers of the same collection and have been assigned to the sameworld-stabilised sub-field; these augmented-reality sound sources arearranged to be collapsed to a single collection-representing soundsource 120 positioned at the top center of the audio sub-field. Otherpositions for the source 120 are, of course, possible such as in linewith the current direction of facing or the location of a particular oneof the sound sources being collapsed.

The collection collapser is further arranged to reverse the collapsingupon receipt of a suitable user command. The collection-representingsound source 120 will generally not be present when the member soundsources of the collection are un-collapsed though it is possible toleave the collection-representing sound source un-muted to serve, forexample, as notification channel to inform the user of events relevantto the collection as a whole.

In a typical implementation, the collection-representing sound source iscreated by the subsystem 13 and is given an ID that indicates itsspecial role; this sound source is then assigned to the same sub-fieldas the collection member sound sources to be collapsed. Thecollection-representing sound source is also given its own audio labelstored in memory 14 with this label being arranged to be temporarilysubstituted for by any notifications generated in relation to thecollection member sound sources (each sound source is also arranged tohave its normal label temporarily replaced by any notification relatedto that source). Whilst the collection member sound sources are notcollapsed, the audibility parameters of these sound sources remain at100% but the collection-representing sound source has its audibilityparameter set to 0% by the collection collapser. However, when thecollection collapser 83 is triggered to collapse the collection membersound sources, these sources have their audibility parameters set to 0%whilst that of the collection-representing source is set to 100% therebyreplacing the collapsed sources with a single sound source emitting thecorresponding audio label (potentially periodically interrupted bynotifications from the services associated with the collapsed sources).On user command, the collapsed sound sources are un-muted and thecollection-representing sound source muted, thereby restoring thecollection to its un-collapsed state.

Rather than the collection changing from its un-collapsed state to itscollapsed state in response to user command, the collection collapsercan be arranged to effect this change automatically—for example, ifthere has been no activity in respect of any member sound source (userservice request/service-originating event notification) for apredetermined period of time, then the collection collapser can bearranged to automatically put the collection in its collapsed state.Similarly, the collection collapser can automatically un-collapse thecollection in response, for example, to the receipt of more than athreshold number of service event notifications within a given time, orupon the user entering a particular environment (in the case of a mobileuser provided with means for detecting the user's environment either bylocation or in some other manner).

To provide clear feedback to the user as to what is occurring when thecollection is being collapsed and un-collapsed, the collection collapseris preferably arranged to change the collection between its two statesnon-instantaneously and with the accompaniment of appropriate audibleeffects. For example, during collapse, the collection-representing soundsource can be faded up as the collection-member sound sources are fadedout. This can be accompanied by a sound such as a sucking in sound toindicate that the member sound sources are notionally being absorbedinto the collection-representing sound source. Alternatively, thelocations of the member sound sources can be moved over a second or twoto the location of the collection-representing sound source. The reverseeffects can be implemented when the collection is un-collapsed.

It may in certain circumstances to have more than onecollection-representing sound source associated with a collection.

As regards the non-collection sound sources (if any) in the audio field,these are typically left un-disturbed by changes in the state of thecollection. However, it would alternatively be possible to arrange forsuch sound sources to be modified to adapt to the presence or absence ofthe collection member sound sources. For example, upon un-collapsing ofthe collection, the location of any sound source close to where a membersound source appears in the audio field can be changed to ensure aminimum separation of sound sources. As another example, uponun-collapsing of the collection the other sound sources can be partiallymuted, at least temporarily.

It will be appreciated that the collection collapser provides more thanjust a way of opening an audio menu where the member sound sourcesrepresent menu list items; in particular, the distribution of thecollection member sound sources in the un-collapsed collection is notconstrained to that of a list but is determined by other considerations(for example, where the sound sources represent augmented realityservices, by the real-world locations of these services).

Unit 84 is a sub-field sound setter intended to set a sounding effectparameter in respect of sound sources of a particular sub-field orsub-fields. The sound setter is operative to set a particular soundingeffect parameter as either on or off for each sound source, whilst thesounding effector 74 is arranged to apply the corresponding sound effectto all sound sources for which the parameter is set to on. Preferably,as default, when the sound setter is enabled the sound sources of allsub-fields have the related sounding effect parameter set to on;however, the user can de-select one or more sub-fields for thistreatment, as desired. In fact, multiple different sound setters 84 canbe provided, each associated with a different sound effect. Typicalsound effects are volume or pitch modulation, frequency shifting,distortion (such as bandwidth limiting or muffling), echo, addition ofnoise or other distinctive sounds, etc.

One reason to employ the sound setter 84 is to make it easy todistinguish one type of service from another or to distinguish thesynthesised sound sources from real sound sources in the environment. Inthis latter case, the audio output devices are, of course, configured topermit the user to hear both real-world sounds as well as thesynthesised sounds.

The user is preferably enabled to choose, via appropriate input means,what sound effect is to be used to make the synthesised sounds distinct;advantageously, the user can also choose to apply or remove the selectedsound effect.

In fact, another way of distinguishing between one group of sounds andanother (such as real and synthesised sounds) is by way of specifying aparticular stabilization for a sub-field(s) containing one of the groupof sound sources to be distinguished. Thus, audio labels foraugmented-reality services can be distinguished from real world soundsby assigning the audio-label sound sources to a head-stabilised field sothat they move relative to the real world as the user turns their head.As another example, the audio labels of general services could beassigned to a head-stabilised sub-field and the audio labels ofaugmented-reality services to a world-stabilised sub-field. As arefinement to always applying the same stabilization to a particularsub-field, the block 26 can be arranged to apply a stabilization schemein which the sub-field is only updated periodically to a specifiedunderlying stabilization, no account being taken between updates of anychanges in orientation of the user's body or head (thereby automaticallyapplying the stabilization associated with the presentation referencevector between updates).

Unit 85 is a range sound setter and is applicable only where an audiosub-field has depth (that is, the range parameter can be different fordifferent sound sources of the sub-field). The range sound setter, whenenabled in respect of a sub-field, is operative, for each sound-sourcein the sub-field, to set a sound source parameter according to the rangeof the sound source. The purpose of doing this is to impart an audiblecharacteristic to the sound source that indicates to the user at least ageneral range of the sound source. This parameter could, for example, bethe audibility parameter with the value of this parameter being set suchthat sound sources at a greater range are presented at a lower volume.However, in a preferred embodiment, the value of the parametercontrolled by unit 85 is used to select which audio label to render froma set of audio labels associated with a sound source, each label havinga different presentation character at least one aspect of which, otherthan or additional to loudness, differs between labels. This aspect is,for example, speaking style, vocabulary, speaker voice, etc. The merechange in a range value included in an announcement is not considered tobe a change in the presentation character of the announcement.

The user can readily learn to associate the differing presentationcharacters with particular range bands. FIG. 16 illustrates an exampleconcerning a sound source for an augmented-reality notification servicefrom the user's local newspaper shop; this service sound source hasthree associated audio labels, stored for it in memory 14, of increasingfamiliarity the closer the sound source is to the user:

Audio label Range >Z2 “Excuse me Sir, would you like your newspaper?”extent Z1-Z2 “Hello Mr Smith, your newspaper” 0-Z1 Hi, John. Paper!”

The unit 85 sets a label-selection parameter for the sound sourceaccording to its range and the relevant label is then used by thespatialisation processor 10. Assuming that the newspaper notificationservice has indicated the real-world location of the newspaper shop tothe apparatus, the processing block 22 can continuously update theposition of the notification-service sound source in the audio field toreflect the movement of the user in the vicinity of the shop. As aresult, the notification audio label will change as the user approachesthe shop (or moves further away). Preferably, of course, thenotification-service sound source is assigned to a world-stabilizedsub-field with the position of the service sound source being set to bein the same direction for the user as the shop itself.

In a variant of the arrangement described above, rather than the soundsources presenting audio labels for services that have associatedreal-world locations, the sound sources can be arranged to present audiolabels for real world entities with real-world locations, the range ofthe sound sources in the audio field being typically, though notnecessarily, set to represent the actual distance between the user andthe real-world location of the entity concerned. Indeed, the concept ofusing announcements each of a different character to indicate distancebetween the user and a sound source can be applied whatever entity, realor virtual, is being represented by the sound source; in this contextthe term “virtual entity” means any non-real-world entity such as aservice, a data item, or application.

The concept of using announcements each of a different character toindicate distance can be further applied to situations beyond thecurrent context of a spatialised audio field. For example, user-carriedequipment can simply be arranged to make a succession of non-spatialisedaudio announcements, each with a differing presentation character, asthe user approaches a particular real-world location or a device inrelation to which range measurements can be made in any suitable manner.

FIG. 17 shows a further example beyond the context of a spatialisedaudio field. In this example, a fixed device 125 with speech outputcapability is arranged to sense the approach of a person 126. As theperson 126 moves closer to the device 125 (the user's movement track isrepresented by dashed line 127 in FIG. 17), the range of the user fromthe device crosses range trigger values Z6, Z5 and Z4 (in decreasingrange order) triggering a respective audio announcement having arange-dependent character. As with the FIG. 16 arrangement, theformality of each announcement decreases with distance (this merelybeing illustrative of one way in which range changes can be indicated tothe person 126). The sensing of the distance between person 126 anddevice 125 can be done in any suitable manner such as by using fixedsensors, round-trip time measurements for signals sent from the deviceand returned by equipment carried by person 126 (with known internalprocessing delay), by a local radio location system interacting withequipment carried by person 126, etc.—in general terms, rangedetermination is done by range-determining equipment at one of theentity, the user, and generally in the environment, either alone or incooperation with auxiliary range-determining equipment at another of theentity, the user, and generally in the environment.

If a data communication path exists between the device 125 and equipmentcarried by the user (for example, via a wireless LAN or a Bluetoothlink), then the announcements made by the device can be pre-specified byperson 126 and sent to the device 125 (together with personal data suchas the person's name). Such a communication path can also be used tosend a range measurement made by the equipment to the device, therebyobviating the need for the latter to make the range measurement.Alternatively, where announcements are held by the person-carriedequipment, range data can be passed from the device 125 to the equipmentto trigger playing of the appropriate announcement by the latter.

Further variants involve announcement data being sent from the device125 to the equipment carried by person 126 for use by that equipment.The sending of this announcement data can be triggered by person 126crossing a range trigger value as measured by device 125 (the data sentbeing for the corresponding announcement); alternatively the appropriateannouncement can be requested from the device 125 as the person-carriedequipment determines that it has crossed a range trigger value. Inanother variant, data on all announcements can be sent from the devicewhen the person is first detected and in this case range-dependenttriggering of the playing of the announcements can be effected based onrange measurements made by either the device, the person-carriedequipment, or a system in the local environment.

Additionally or alternatively to the announcements being made whentriggered by a range trigger value being reached, the announcements canbe made at periodic intervals, the announcement used being dependent onthe current range between user and the device 125.

In the foregoing examples related to FIG. 17, where the device 125announces its presence through announcements made by the user-carriedequipment, this latter can be understood as acting as a proxy for thedevice 125 (regardless of whether the announcement phrasing is infirst-person device-related terms or in third person terms). Rather thanhaving user-carried equipment act a proxy for device 125, equipment(typically fixed) in the local environment but not specific to thedevice 125, can be arranged to act as an announcement proxy for thedevice. In this latter case, the announcement (stored in one of thelocal-environment equipment, user-carried equipment, and the device 125,and retrieved to the local-environment equipment as required) ispreferably made either without any specific directional character orsuch as to appear to the user to be coming from the device 125 itself(which is more complex to achieve as this approach needs to know theuser's location relative to the equipment and to adapt to changes inthis location as the user moves). As already indicated above, equipmentin the local environment can also be used to determine the range betweenthe user and device 125 in which case it can additionally be used todetermine the appropriate announcement and either retrieve (and use) ititself or inform the device 125 or user-carried equipment (which ever isto make the announcement) which announcement to use.

As an alternative to storing multiple announcements each with adifferent presentation character and selecting the announcementappropriate for the current range value, a single announcement can bestored to which a presentation character appropriate to the currentrange is applied—for example, where the announcement is stored as textdata for conversion to speech via text-to-speech converter, the voicedata used by the text-to-speech converter can be selected according torange so that the voice in which the announcement is made changes withrange.

Selecting a Sound Source in the Audio Field

A variety of different techniques can be used to select a particularsound source from those present in an audio field generated by the firstor second apparatus described above. Three specific selection techniqueswill now be described with reference to FIG. 18 which shows furtherdetail of the second apparatus (though it is to be understood that thetechniques are equally applicable to the first apparatus); the generalcharacter of each of the selection techniques to be described is asfollows:

-   1.)—rotation/displacement of the audio field to bring the sound    source to be selected to a particular selection direction with    respect to the user;-   2.)—moving an audio cursor to coincide with the sound source to be    selected;-   3.)—speech input with restricted recogniser search space.

It will be appreciated that the apparatus need only be provided with oneselection technique although providing alternative techniques adds tothe versatility of the apparatus.

With respect to the first technique, it is convenient to define aselection direction as being the horizontal straight-ahead facingdirection of the user, though any other convenient direction could bechosen such as the actual current facing direction or that of thepresentation reference vector. An indication of the chosen selectiondirection is supplied on input 135 to block 26 (this input 135, but notthe block 26, is shown in FIG. 18). As already described, the user canrotate/displace the audio field by inputs to block 26 (on input 28 shownin FIG. 10), these inputs being generated by input device 136 (FIG. 18).This input device can take any suitable form, for example, amanually-operable device or a voice-input device set to recogniseappropriate commands. For a 2D spherical field, the apparatus isarranged to permit control of both the azimuth angle and elevation angleof the audio-field reference vector relative to the presentationreference vector; for a 2D cylindrical field, the apparatus is set topermit control both of the azimuth angle of the field and of its height(elevation). This permits any point (and thus any sound source) in thefield to be brought into line with the predetermined selection directionby rotations/displacement commanded by input device 136.

A selection-direction comparison unit 137 of the source parameterset/modify block 70 is fed with an input 138 from block 26 indicatingthe angular offset between the selection direction and the presentationreference direction (this offset is readily determined by block 26 fromthe inputs it receives). Given this information, unit 137 determines ifany sound source in the audio field lies in the selection direction (orwithin a defined angular distance of it) and, if so, sets a selectionparameter of that sound source to ‘true’, resetting the parameter to‘false’ upon the sound source ceasing to be in alignment with theselection direction. The unit 137 operates on basis of the renderingposition of each sound source after any processing by other units ofblock 70 that may affect the rendering position of that sound source.The unit 137 may also set a sounding effect parameter for the soundsource to give a distinctive sound for that source in order to indicateto the user when a sound source lies in the selection direction.

The input device 136 as well as enabling the user to rotate/displace theaudio field, also enables the user to indicate that a sound source lyingin the selection direction is to be selected. This indication isgenerated, for example, using a selection button or upon recognition ofa command word such as ‘select’, and results in a corresponding signalbeing fed on line 139 to a mode and source control block 128 of theoutput selection block 12. On receiving this signal, block 128 accessesthe memory 15 to determine which sound source, if any, currently has itsselection parameter set to ‘true’; provided such a source is identified,the block 128 switches the apparatus from its desktop mode to itsservice mode and instructs the spatialisation processor 10 on line 129to output a full service feed for the identified service sound source.

It may be noted that when the apparatus is in its desktop mode, at anygiven moment some of the sound sources may be in a fully muted state dueto operation of units of the source parameter set/modify block 70. Sinceit is unlikely that a user will intentionally be trying to select such amuted source, when the mode and source control block 128 accesses memory15 to identify a sound source lying in the selection direction, it ispreferably arranged to ignore any muted sound source, notwithstandingthat the source lies in the selection direction.

The fact that the FIG. 10 permits the presence of multiple sub-fieldshas two consequences for the above-described selection technique.Firstly, it will generally be desirable for the input device 136 to beable to rotate/displace any desired one of the sub-fields independentlyof the others; however, when the user wishes to move a sound source tolie in the selection direction, it is simplest to arrange for allsub-fields to be moved together by device 136. Secondly, with multiplesub fields that are independently movable, it is possible that multiplesound sources can lie in the selection direction at the same time; inorder to cope with this, block 128 can operate any suitableprioritisation scheme to choose between such sound sources or canpresent the choice of sources to the user to allow the user to selectthe desired one of the sources lying in the selection direction.

With regard to the selection direction comparator unit 137 setting asounding effect parameter to give an audible indication to the user whena sound source lies in the selection direction, the operation of unit137 can be refined also to adjust a sounding effect parameter toindicate when a sound source is near the selection direction, theadjustment to the sound effect being such as to provide an indication ofthe direction in which the sound source needs to be moved to come intoalignment with the selection direction.

The second selection technique to be described uses an audio cursor.This cursor is a special sound source that is arranged to berotated/displaced by a cursor control input device 140 which, like inputdevice 136, can take any suitable form; indeed, devices 136 and 140 canbe combined with a mode control for switching between the respectivefunctions of the two devices. For the FIG. 10 apparatus, onestraight-forward way of implementing the audio cursor is as a soundsource aligned with the audio-field reference vector of a dedicatedsub-field; in this case, the output of the cursor control input deviceis fed to block 26 to rotate/displace that sub-field (from which it canbe readily seen that the function of input device 140 can easily beeffected by input device 136). Preferably, the audio-cursor sub-field isarranged not to move with the other sub-fields and to be bodystabilised. An alternative audio cursor implementation is for the inputdevice 140 to directly set the position of the audio-cursor sound sourcerelative to the presentation reference vector, this being theimplementation depicted in FIG. 18 where a block 141 uses the outputfrom device 140 to calculate the current cursor position. With eitherimplementation, the current rendering position of the cursor is fed tothe source parameter set/modify block 70 where it is stored in a memory144.

A cursor sound setter unit 145 of block 70 compares the position of thecursor against the final rendering position of each sound source (theunit 145, like the unit 137, is thus arranged to operate using therendering position of each sound source after any processing by otherunits of block 70 that may affect the rendering position of that soundsource). If no sound source is close to the cursor's current position, acursor-sound parameter is set to a corresponding value and is passed,along with the cursor ID and rendering position, via the converter 66 tomemory 15. The spatialisation processor, in conjunction with soundeffector 74, then causes a distinctive cursor sound to be generated atthe appropriate position in the audio field, the nature of the soundbeing such as to indicate to the user that the cursor is not close toanother sound source. The sounding effector 74 is preferably arranged toprovide the cursor sound without the need to refer to the subsystem 13,this variation from the treatment of the cursor as the other soundsources being justified by the special status of the cursor soundsource.

Upon the unit 145 determining that the cursor is close to a sound source(that is, within a threshold distance which is preferably settable bythe user), it sets the cursor-sound parameter for the cursor to indicatethis for example by setting it to a value that is dependent on thedirection of the source relative to the cursor. The sounding effector 74then causes the cursor sound to be correspondingly adapted to indicatethis relative direction to the user, for example:

Relative Positions Cursor Sound Sound Source above cursor Alternatinghigh-frequency dots and dashes Sound Source below cursor Alternatinglow-frequency dots and dashes Sound Source to left of cursorMiddle-frequency dots Sound Source to right of cursor Middle-frequencydashes

As an alternative, appropriate words could be used (‘above’, ‘below’,‘left’, ‘right’) repeated at a low volume level.

The distance between a sound source and the cursor can also be indicatedaudibly such that it is possible to tell whether the cursor is gettingcloser to, or further from, the sound source. Thus, in the case of theabove example using dots and dashes, the repetition rate of the dots anddashes can be increased as the cursor moves closer to a sound source anddecrease as the cursor moves away; alternatively, the separationdistance can be indicated by appropriate words.

Thus, in general terms, the cursor sounds are modified to provide anaudible indication of when the cursor is close to a sound source withthis indication being preferably set to indicate the distance and/ordirection of the sound source.

When the cursor coincides with a sound source (at least in terms oftheir direction from a user reference location), the unit 145 sets thecursor-sound parameter to a further value which the sounding effector 74translates to another unique sound such as rapid beeping. Unit 145 alsosets to ‘true’ a selection parameter of the sound source to indicate itscoincidence with the cursor. If the user now indicates, using inputdevice 140, that the sound source is to be selected, a correspondingsignal is sent on line 142 to the mode and source control block 128. Aswith the first selection technique, this causes block 128 to accessmemory 15 to determine which sound source has its selection parameterset to ‘true’ before switching the apparatus to its service mode inwhich a full service feed of the selected service sound source isenabled.

The block 128 can be arranged to handle muted sources and multiplesources at the cursor position in the same way as it handled thecorresponding situations for the first selection technique.

The unit 145 can be arranged not only to set the selection parameter ofthe sound source pointed to by the cursor, but also to set the value ofa sounding effect parameter of any sound source determined by unit to beclose to, or in line with, the audio cursor so that the sounds emanatingfrom that sound source are adapted by the sounding effector 74(including, potentially by the adding in of extra sounds or words) toindicate the closeness (and, optionally, distance to) the audio cursor;thus, for example, the volume or pitch of the sound source, or thedegree of application of a vibrato or echo effect to the sound source,could be increased as the cursor approached the sound source (anddecreased as their separation increased). The relative direction of thecursor from the sound source (or the reverse direction) can also beindicated by sounds or words output from the sound source. Thus, ingeneral terms, the sounds emanating from the sound source are modifiedto also provide an audible indication of when the cursor is close to thesound source with this indication being preferably set to indicate thedistance and/or direction of the sound source.

The foregoing modification of sounds emanating from a sound source nearthe cursor can be done as an alternative to, or additionally to, settingthe cursor-sound parameter to indicate sound-source closenessdistance/direction.; in other words, the audible indication producedwhen the cursor is close to a sound source can be provided via thecursor and/or the sound source. As an example of providing a respectivecomponent of this audible indication from the sound source and thecursor, the sounds emanating from the sound source can be modified toindicate the proximity of the cursor and their separation distance,whilst the cursor sound source can be used to indicate the direction ofthe sound source; with this arrangement, where there are several soundsources within the closeness threshold of the cursor, the sound sourcesindicate this closeness by the sounds they emit whilst the cursorindicates the direction to the closest sound source.

Where the audio sub-fields are of 3D form, it is possible to arrange forthe audio cursor to be moved in the third (range) dimension. This canmost conveniently be done where, as shown in FIG. 18, the cursor-controlinput device 140 is used to directly set the cursor position relative tothe presentation reference vector; in this case, the input device issimply further arranged to set the range of the audio cursor and thisrange value is stored in memory 144. In order to provide the user withan indication of the range of the audio cursor, the cursor sound setterunit 145 is preferably arranged to set the value of a sounding effectparameter of the cursor according to the current range of the cursor(regardless of the proximity of any sound source), the sounding effector74 then producing a correspondingly modified sound for the cursor. Forexample, where the sounding effector produces a tone to represent thecursor, the volume of the tone can be adjusted, via an audibilityparameter, to reflect the current range position of the cursor (the-greater the range, the quieter the cursor sounds). Alternatively, thefrequency of the cursor tone can be varied with the current range of thecursor.

It may be noted that the focus expander 80 can conveniently be linked tothe audio cursor to expand the region of the audio field about thecursor rather than about the current direction of facing of the user aswas earlier described. In this case, the unit 80 is supplied with thecurrent cursor position from memory 144 rather than with the currentfacing direction of the user.

The third selection technique is based on the use of a speech recogniser150 to determine when the user is speaking the sound label of a soundsource, the speaking of such a label being taken to be an indicationthat the user wishes to select the source.

Speech recogniser 150 has speech input 151 and associated vocabulariesthat define the words between which the recogniser is to distinguish. Inthe present case, the vocabularies associated with the speech recogniserinclude a command vocabulary (stored in memory 152) holding commandwords such as “desktop” (to return to the desktop mode); “louder” and“softer” (to generally increase and decrease volume levels); “rotateleft”, “rotate right”, “up”, “down” (where sub-field rotation is to beeffected by spoken command), numbers 1 to 10 (to identify sub-fields),etc. The audio labels held in memory 14 also define a vocabulary for therecogniser, the phonetic contents of the label words being madeavailable to the recogniser through an appropriate reference database(not shown). In the event that a sound source has its associated labelconstituted by an audio feed from the source or by non-word sounds, thenthe label memory is preferably arranged to store appropriate words thatthe user might use to select the source, these words beingadvantageously supplied by the related service when first selected bysubsystem 13.

In order to facilitate the operation of the speech recogniser 150,various measures can be taken to the reduce the search space of therecogniser (that is, the range of words with which it tries to match aspoken word received via input 151). In the present case, threedifferent restrictions are applied to the search space though it is tobe understood that these restrictions can equally be applied inisolation of each other. These restrictions are:

-   (i) A restriction to sound sources positioned within a range gate    determined by the loudness of the spoken input (this restriction is    only relevant where the audio sub-field(s) have depth—that is, a    spread of range values). Assuming that the user knows the general    range of the sound source the user wishes to select, then the user    can speak the audio label of the source at a loudness volume    reflecting the range of the source. Typically, the user will speak    the label of a nearby source louder than that of a more distant    one—the underlying model here is that the user is reflecting the    fact that nearby sound sources are generally louder a the user than    far away ones. However, it would also be possible to use the    opposite scheme where the user speaks louder for further away    sources—here the underlying model is that the user needs to speak    louder in order for the remote source to ‘hear’. The loudness of the    speech input is measured by block 154 and converted to a range gate.    FIG. 19 shows an example relationship between loudness and range    that can be used by block 154; in this case, for a received loudness    of L1, a range gate G is determined corresponding to equal    increments ΔL either side of L1. The derived range gate G is passed    to a restrictions application block 155 that accesses memory 15 to    determine which sound sources lie within this range gate. The    recogniser search space is then restricted to the labels (or other    identification words) associated with the sound sources within the    range gate. To help the user speak a label at the correct loudness,    it is possible to provide a calibration mode of operation (selected    in any suitable manner) in which when a user speaks a word, that    word (or another sound) is rendered in the audio field at a range    corresponding to that assessed by the loudness-to-range classifier    154; the implementation of this feature is straight-forward and will    not be described in further detail.-   (ii) A restriction to sound sources that are currently audible. This    restriction is implemented by block 155 which accesses memory to    determine whether the current value of the audibility parameter of    each sound source is such as to permit it to be heard. The    recogniser search space is then restricted to the labels (or other    identification words) of the currently audible sound sources. It is    also possible to arrange for sound sources having reduced audibility    (that is, sources muted to at least predetermined degree) to be    discarded.-   (iii) A restriction to sound sources that lie in the general facing    direction of the user. To implement this restriction, the    restriction application block 155 is supplied on input 156 with the    current facing direction of the user, this direction being supplied    by block 26 and specifying the current facing direction relative to    the presentation reference vector. Block 155 then searches memory    for sound sources lying within a predetermined angular extent of the    facing direction (it should be noted that the facing direction    supplied to block 155 should first be converted to the same    coordinate scheme as applied by converter 66 to the sound source    rendering positions). After determining which sound sources lie in    the general direction of facing of the user, the block causes the    recogniser to restrict its search space to the labels (or other    identification words) associated with these sound sources.

Whilst the foregoing assumes that words will be used to identify soundsources, it is also possible to alternatively and/or additionally usespecific sounds (such as whistling, clicking, grunts, laughter, humming,etc.) which the recogniser 150 would be set to recognise.

It will be appreciated that although user speech input has beendescribed above in relation to selecting a particular service via itsaudio label, it is also possible to use speech input to address theservice in the service mode of the apparatus (and, indeed, it is alsopossible to arrange for a service to be addressed and provided withinput whilst the apparatus is still in its desktop mode—in this case,addressing a service by speaking its audio label is not assumed to be anindication that full service feed of that service is required, thisrequiring an additional pre- or post input such as speaking the word“select”).

It may also be noted that restricting the speech recogniser search spaceby excluding the labels associated with services lying outside a rangegate indicated by the loudness of the user input, can be used not onlywith user interfaces where the services are represented through soundsources in an audio field, but also generally with any user interfacewhere items are represented to a user with a perceivable range value andthe items have respective associated labels by which they can beaddressed. For example, items can be presented on a visual display withthe range value of each item being perceivable either by perspective inthe visible image or from an associated text label.

It will be appreciated that other techniques additional to thosedescribed above can be used for selecting a particular sound source inthe spatialized audio field. For example, a point-by-hand interface canbe employed in which the user's pointing gestures are detected (forexample by sensing changes in an electric field or by interpreting astereo image) and used to determine which spatialized sound source isbeing indicated.

Manually-Operated Input Devices

FIGS. 20 to 24 show various forms of manually-operated input device thatcan be used for input device 136 or 140 of FIG. 18.

FIG. 20 illustrates an input device 160 similar in form to knowntrackball devices and comprising trackball 161 the rotation of which ismeasured by sensors (not shown) about two orthogonal axes. The inputdevice 160 is particularly suited for controlling field rotation andaudio cursor movement in the case of a spherical audio field, althoughit can also be used with other forms of audio field.

Conventional trackball devices measure trackball rotation about two axeslying in a horizontal plane (assuming the mounting plane for thetrackball to be horizontal). This initially appears inappropriate for adevice intended to control rotation of a spherical audio field inazimuth and elevation, rotation in azimuth being about a vertical axisand therefore not directly capable of imitation by a conventionaltrackball device. Accordingly, it is envisaged that embodiments ofdevice 140 provide for measuring rotation about vertical axis 164 aswell as about a horizontal axis such as axis 162.

However, it has been found that having the trackball 161 rotatable aboutthe same axes as a spherical audio field it is intended to control hascertain drawbacks. In particular, rotating the trackball about avertical axis is not a very natural action for the user. Furthermore,where, as in embodiments to be described below, rotations of thetrackball are arranged to produce rotations of the same angular extentof the audio field so that the surface of the trackball can be markedwith indications of the current orientation of the audio field, havingthe straight-ahead position lying at the mid-height of the trackballand, as a result, not clearly visible to the user, is not helpful intranslating the indications carried by the trackball into informationrelevant to using the audio field. As a consequence, it is an acceptablecompromise to measure the rotation of the trackball about its twohorizontal axes 162 and 163 with rotation about the axis 163 being takenas indicating the required azimuth rotation (rotation in elevation beingindicated by rotation about axis 162).

By the use of appropriate rotation sensing arrangements, it is possibleto sense the current orientation of the trackball 161 and then orientatethe audio field to the same orientation; one suitable sensingarrangement involves providing a pattern of markings (not necessarilyhuman visible) on the surface of the trackball such that reading anysmall area of the pattern opposite a small sensing camera (or otherappropriate sensor depending on the nature of the markings) issufficient to uniquely determine the orientation of the trackball. Thispermits the trackball to be marked in a human visible manner to indicateto the user the current orientation of the trackball and thus thecommanded rotation of the audio field—where no stabilisation offset isapplied by block 26, this orientation directly corresponds to that ofthe audio field relative to the presentation reference vector (thiswould be the case, for example, where headphones are being used and theaudio field is head-stabilised). By way of example, the eight quadrantsof the trackball can each be given a respective colour with theaforesaid sensing pattern being marked out using infrared or magneticinks; FIG. 20 depicts the application of different markings (such ascolours) to different quadrants with three such quadrants 166, 167, and168 being visible.

Directly marking the outside of the trackball to indicate orientationhas a disadvantage in that if the trackball 161 is allowed to berotatable about all three axes 162-164, then rotations about all axesmust be measured and corresponding rotations effected to the audiofield—if this is not done, the markings on the trackball will quicklycease to correspond to the orientation of the audio field. Whilst it ispossible to engineer restrictions on the rotation of the trackball sothat it can only rotate about the two desired axes, an alternative andpreferred approach is to provide a visual orientation indicatorarrangement that uses the sensed rotation of the trackball to determinethe orientation to be indicated by the arrangement. Such an arrangementavoids the need to match the orientation of the trackball with that ofthe audio sphere and it is possible to use a conventional two-axisrotation sensing arrangement that simply measures angular changes(rather than absolute orientations) potentially with slippage.

One suitable form of fixed visual orientation indicator arrangement isillustrated in FIG. 21 that shows a trackball-based input device 170similar to that of FIG. 20 but without quadrant markings on the surfaceof its trackball 171; instead, a row of indicator lights 173 (typicallyLEDs) is provided. Each LED 173 represents a respective quadrant of theaudio field, the quadrant concerned being depicted, for example, by agraphic adjacent the LED. The activation of the LEDs is controlled toindicate the current commanded orientation of the audio field as knownto block 26 of the FIG. 10 apparatus. Thus, as a commanded rotation ofthe audio field brings the presentation reference vector within aquadrant of the audio field (assuming, for the moment, no stabilisationrotation of the audio field), the block causes the LED 173 correspondingto that quadrant to be activated, all other LEDs being deactivated.

Rather than arranging the LEDs 173 in a row, different coloured LEDs (orother light emitting devices) could be grouped together inside thetrackball itself, the latter being translucent or transparent so theuser can see the colour of the currently activated LED and thereforegain an indication of the current orientation of the audio sphere. Thislatter configuration requires an appropriate arrangement for poweringthe LEDs inside the trackball and this can be achieved either by anarrangement of sliding contacts or by flexible wiring runs and physicallimiters on the movement of the trackball to prevent excessive twistingof the wiring. In a further alternative embodiment of the indicatorarrangement, the trackball surface is covered with a layer the visualproperties of which can be altered by control signals; in this mannerthe visual appearance of the trackball provides the desired orientationindication.

Rather than the visual orientation indicator arrangement indicating theorientation of the audio field relative to the presentation referencevector without regard to any stabilisation rotation of the audio field(that is, only indicating the commanded rotation), it is preferable toarrange for the indicator arrangement to indicate the audio-fieldorientation relative to a selected “indicator reference” direction (forexample, the presentation reference vector, the current facing directionof the user, the forward-facing direction of the user, a world-fixeddirection such as North, or a vehicle straight-ahead direction forin-vehicle audio systems) with account being taken, where required, ofany rotation of the audio field effected to give it a specifiedstabilisation. The required output indication from the indicatorarrangement is determined, for example, by block 26 and may requireinformation (rotation of the user's head relative to their body,rotation of the user's head relative to the world or to a vehicle,rotation of the user's body relative to the world or to a vehicle) notavailable from any sensors currently being used for achieving aspecified audio-field stabilisation sensors—in such cases, theappropriate sensors will need to be provided to supply the requiredinformation to the block 26.

Basically, in order for the block 26 (or other processing means) toappropriately control the visual orientation indicator arrangement, itneeds to know about any changes in the offset between the audio fieldreference and the presentation reference vector (either user commandedor required to achieve a particular stabilisation), as well as anychanges in the orientation of the indicator reference direction relativeto the presentation reference (caused, for example, by rotation of theuser's head or body). In certain cases, at least components of thechanges in the offset between the audio field reference and thepresentation reference vector required to achieve a particularstabilisation in the presence of rotation of the user's head/body, willmatch the changes in orientation of the indicator reference relative tothe presentation reference resulting from the rotation of the user'shead/body. In such cases, it is only necessary to take account of theunmatched components (notably, but not in all cases exclusively, theuser-commanded component) of the offset between the audio fieldreference and the presentation reference. In implementing block 26 (orother processing means) for determining the orientation between theaudio-field reference and the indicator reference direction, it is not,of course, necessary first to determine the offset between the audiofield reference and the presentation reference vector and theorientation of the indicator reference relative to the presentationreference, before going on to determine the orientation between theaudio-field reference and the indicator reference direction; instead thevarious measured components can be directly combined to determine theorientation between the audio-field reference and the indicatorreference direction (with components that match each other outpreferably not being processed). This is depicted in FIG. 22 where block26 is shown as having a processing sub-block 177 for determining theoffset between the audio-field reference and the presentation reference,and a processing sub-block 178 for determining the orientation betweenthe audio-field reference and the indicator reference direction, eachsub-block working directly from measured components (for example:commanded rotation, rotation of user's head relative to user's body, androtation of user's body relative to the world—from which rotation of theuser's head relative to the world can be derived; it will be appreciatedthat this latter could be measured, in which case one of the othermeasured components—not commanded input—is no longer needed). Sub-block178 controls a visual orientation indicator arrangement 179.

The table below indicates for audio output devices in the form ofheadphones (inherently head-stabilised), the component quantities neededto be known, for each of three different stabilisations, in order todetermine the orientation of the audio field relative to each of threedifferent indicator reference directions.

Orientation of Audio-Field Stabilisation Indicator Reference w.r.t.Indicator Reference Head Stabilised Current facing direction Commandedrotation (inherent) (presentation reference) Forward facing directionCommanded rotation + head rotation (wrt body)¹ World direction Commandedrotation + Head rotation (wrt world)¹ Body Stabilised Current facingDirection Commanded rotation − (presentation reference) head rotation(wrt body) Forward facing direction Commanded rotation World directionCommanded rotation + Body rotation (wrt world)¹ World Stabilised Currentfacing direction Commanded rotation − (presentation reference) Headrotation (wrt world) Forward facing direction Commanded rotation − Bodyrotation (wrt world)^(1,2) World direction Commanded rotation ¹Requiressensing additional to that needed for stabilisation ²In this case, anycomponent of the offset between the audio-field reference and thepresentation reference that is due to rotation of the user's headrelative to the user's body is matched by a change in orientation of theindicator reference direction relative to the presentation reference,thereby leaving the offset components of the user-commanded rotation androtation of the user's body relative to the world.

In one preferred embodiment, the audio field is body-stabilised and theindicator reference direction is the forward-facing direction of theuser.

Similar tables can readily be produced for body-mounted,vehicle-mounted, and world-mounted audio output devices. Also, thetables can be extended to include vehicle-stabilised audio fields and anindicator reference direction of a vehicle straight-ahead direction.

It will be appreciated that embodiments of the visual orientationindicator arrangement that indicate the current orientation of the audiofield relative to a specified indicator reference direction as describedabove, facilitate an appreciation by the user what part of the audiofield they are currently looking at and enables them to more rapidlyfind a desired service sound source. It will also be appreciated thatthe visual orientation indicator arrangement may change the indicatedaudio-field orientation without any operation of the trackball if theorientation of the user changes and results in audio-field rotationrelative to the indicator reference direction as a consequence of thecurrent audio field stabilisation.

The LEDs 173 can also be used to indicate when a new service soundsource appears within a quadrant and/or when a service sound source in aquadrant has a new notification. In either case, the LED for thequadrant in which the service sound source lies can be arranged to flashat least for a limited period. If the LED concerned is already activatedbecause it encompasses the selected direction controlling LEDactivation, then the LED can still be flashed to provide the requiredindication. It is, of course, possible to provide a separate set of LEDs(or other visual indicators) solely for the purpose of indicating a newsource or new notification in which case the required indication cansimply be activation of the relevant LED. A set of LEDs can be providedfor this purpose in device 160 of FIG. 20.

Another suitable form of fixed visual orientation indicator arrangementis illustrated in FIG. 23 that shows a trackball-based input device 180in which a small display panel 185 is mounted to show a depiction ofthat part of the audio field lying either side of the indicatorreference direction. This depiction preferably gives both an indicationof the portion of the audio field concerned (for example, in terms offield coordinate ranges, or a quadrant name), and an indication of thesound sources in this portion of the audio field. The orientation of theaudio field can be indicated by other types of diagram or imagedisplayed on display panel 185.

The FIG. 23 input device also includes, as well as a trackball 181, aset of LEDS for indicating, in the manner described above with referenceto FIG. 21, when a new sound source or new notification is available.

FIG. 24 shows a form of input device 190 specifically adapted for usewith cylindrical audio fields though also usable with other fields. Theinput device 190 comprises a cylinder 191 that can be moved by hand backand forth along a shaft 192 coaxial with cylinder 191 (see dashed arrow193) as well as rotated (see dashed arrow 194) about the shaft. Both theposition of the cylinder 191 along the shaft 192 and the angularposition of the cylinder 191 about the shaft are measured by suitablesensor arrangements (for example, electro-optical sensors) and arerespectively used to set the height and azimuth angle of the cylindricalfield being controlled. The cylinder 191 carries an index marking 195that cooperates with a fixed scale 196 to indicate the current height ofthe audio field. Further markings (not shown) on the cylinder can beused to indicate the current azimuth setting of the audio field. A setof LEDs 198 (or other light output devices) can be used to indicate thepresence of a new sound source or of a new notification, the LED 198activated being dependent on the height of the sound source concerned(the scale 196, or other markings, can be used to indicate the heightsignificance of each LED).

With the form of the input device 190 shown in FIG. 24, because theazimuth orientation of the audio field is indicated by markings carriedby the cylinder 191, only the offset between the audio-field referenceand presentation reference can be indicated and this without any accountbeing taken of rotation of the audio field to achieve a particular fieldstabilisation. To overcome these limitations, the input device 190 canbe provided with any of the above-described forms of visual orientationindicator arrangements controlled by block 26 to give the fieldorientation relative to a given indicator reference direction.

It will be appreciated that the above-described forms of visualorientation indicator arrangements controlled by block 26 (or otherprocessing means) to give the field orientation relative to a givenindicator reference direction, can be implemented separately from theinput devices themselves. Furthermore, the visual orientation indicatorarrangements can still be employed where the user is not provided withmeans to change the offset between the audio field reference and thepresentation reference (though, of course, there is little point indoing this in the above-mentioned cases where the user-commanded inputwas the only variable component of the orientation of the audio fieldreference relative to the indicator reference). Finally, it may be notedthat the orientation of the audio-field reference relative to theindicator reference may have one, two or more degrees of freedom and thevisual orientation indicator arrangement is therefore preferablycorrespondingly adapted to be able to indicate all degrees oforientation changes. By way of example, where a head-stabilised audiofield is presented through headphones and the indicator referencedirection is the current facing direction, then if only azimuth changesare involved for user-commanded rotations, for audio-field stabilisationand in determining the current orientation of the indicator referencerelative to the audio field, then the orientation of the audio fieldrelative to the indicator reference has only a single degree of freedom;however, if, for example, the user-commanded inputs can also change theelevation between the audio field reference and the presentationreference, then the orientation of the audio field relative to theindicator reference will have two degrees of freedom. The visualorientation indicator arrangement can, however, be restricted toindicate less than all of the degrees of freedom associated with theorientation of the audio field relative to the indicator reference.

Each of the input devices 160, 170, 180 and 190 also includes aselection button, respectively 165, 172, 182, and 197 for enabling theuser to indicate that they wish to select a particular service eitherlying in the selection direction or overlaid with the audio cursor.Where sub field rotation/displacement (including rotation/displacementof a cursor sub-field) is to be controlled by any of the devices, thenthat device is preferably also provided with means for selecting whichsub field is to be controlled; these means can take any suitable formsuch as selection buttons, a rotary selector switch, a touch screenselection display, etc. Similarly, selection means can be provided toswitch between audio (sub-)field control and cursor control where thecursor, instead of being associated with a sub-field, has its renderingposition directly controlled by the input device. Further selectionmeans can be provided to enable a user to select a particular indicatorreference direction from a set of such directions which block 26 is setup to handle.

The input devices described above are suitable for use with 2D audiofields. The devices are also suitable for 3D audio fields where thefield/audio cursor is not required to be moved in the third (range)dimension. Where exploration in the third dimension is required (such aswhen an audio cursor is to be moved back and forth in the Z or rangedimension), each device can be provided with a range slider generatingan output signal in dependence on the position of a slider along atrack.

Variants

It will be appreciated that many variants are possible to the abovedescribed embodiments of the invention. For example, in relation to thecylindrical audio field forms described above, whilst these have beendescribed with the axis of the cylindrical field in a verticalorientation, other orientations of this axis are possible such ashorizontal. Also with respect to the cylindrical field form embodiments,it is possible to implement such embodiments without the use of leakageinto the focus zone and, indeed, in appropriate circumstances, evenwithout the use of a focus zone.

As regards the audio labels used to announce each service sound sourcein the desktop mode of the described apparatus, these labels can includea component that is dynamically determined to indicate the actual orrelative position of the corresponding sound sources in the audio field.Thus, if an email service is provided on the second floor of an audiofield organised as depicted in FIG. 8, then the audio label could be“email on second” or “email down one” (where the user is currentlylocated on the third floor). As another example, the audio label of aservice sound source can include the word “left” or “right” to indicatewhether the service is to the left or right of the user. Thus, a servicesound source may indicate its location as “upper left” when situated tothe left and above the reference direction being used. In oneimplementation of this feature, a dynamic label processor continuallychecks the position of each sound source (either its absolute positionin the audio field or its position relative to a selected reference suchas the user's current facing direction, or straight-ahead facingdirection, or the presentation reference) and updates the audio label ofthe sound source accordingly in memory 14. In an alternativeimplementation, the sounding effector 74 (see FIG. 10) is arranged toadd an appropriate location key word(s) to each label according to thevalue of a location parameter that is set for each sound source by alocation-label setter of the source parameter set/modify block 70. Thislocation-label unit functions by examining the position of each soundsource at frequent intervals and determining the appropriate locationkeyword(s) to add to its audio label depending on the absolute orrelative position of the sound source (again, relative position can bejudged in relation to any appropriate reference such as user currentfacing direction, straight-ahead facing direction, or presentationreference). As regards the details of determining the location of asound source relative to the selected reference, this is similar to theabove-described determination of the orientation of the audio-fieldreference relative to the indicator reference for controlling a visualorientation display arrangement; however, a further, possibly variable,component, is now involved, namely the location of the sound sourcerelative to the audio-field reference. Whilst the location of a soundsource relative to the selected reference may have two or more degreesof freedom, in some embodiments it may be appropriate to restrictdetermination of this relative location to only one of the degrees offreedom, the audio indication of this relative location being similarlylimited.

The possibility of having multiple sound sources associated with aservice has been generally described above. One example where this canbe useful is in relation to a service such as electronic mail or voicemail where it is desired to be able to directly select either the mailinbox or outbox (or message generation function).; in this case, each ofthese service elements is represented by a corresponding sound source inthe desktop audio field.

Another example of the use of multiple sound sources associated with thesame service was given above in relation to the ghost advisory serviceused to provide upper and lower summary sound sources 60, 61 (see FIG. 8and related description). The advisory service is a ghost service in thesense that its only manifestation is through the audio labels associatedwith its sound sources—there is no underlying service component that canbe activated by selection of the sound sources.

A further example of a ghost service with multiple sound sources is theuse of a sub-field to provide an audio compass available to the userindependently of whatever other audio sub-fields are being provided. Thecompass sub-field takes the form of a world-stabilised sub-field withone or more sound sources at key compass points (such as north, south,east and west, and the user's current facing direction). An electroniccompass can be used to provide the necessary input to block 26 to rotatethe audio sub-field such that the spatialized north sound source alwayslay in the north direction relative to the user (the other key compasspoint sound sources, being then automatically correctly aligned as aresult of their positioning in the audio field relative to the northsound source). The compass-point sound sources can be set to announcecontinually or, where speech command input is provided, only when acommand (such as “Compass”) is spoken. Similarly, the user's currentfacing direction can be arranged to be announced upon the user issuing acommand such as “Direction”. Whilst the accuracy of perception by theuser of the key compass points announced through the spatialized soundsources will only be very approximate, the announcement of the currentfacing direction can give the user much more precise directioninformation since it announces a measured direction rather than relyingon spatial audio awareness to convey the direction information.

Of course, the audio compass can also be implemented where only asingle, world-stabilised audio field is produced by the apparatus.Furthermore, additional useful functionality can be achieved by linkingthe apparatus with an electronic map system that has an associatedabsolute position determining system such as a GPS system. In this case,the user can specify a map location (for example, by pointing to itwhere the electronic map system has an appropriate display subsystem fordetecting which map location is being pointed to) and a sound source isthen automatically generated in the audio field in alignment with thedirection of the map location indicated. This sound source can output anaudio label giving information about what is at the map location andalso give instructions as to whether the user needs to turn their headleft or right to look directly in the direction of the map location.Another possible function would be to tell the user what is ahead intheir current facing direction or current direction of travel.

It will be appreciated that most of the functionality of the functionalblocks of the various forms of apparatus described above, will typicallybe implemented in software for controlling one or more general-purposeor specialised processors according to modem programming techniques.Furthermore, whilst a number of separate memories have been illustratedthe described embodiments, it will be appreciated that this is done tofacilitate a clear description of the operation of the apparatus; memoryorganisations and data structures different to those described aboveare, of course, possible.

It should also be understood that the term “services” as used above hasbeen used very broadly to cover any resource item that it may be usefulto indicate to the user in much the same way as a PC visual desktop canbe used to represent by visible icons a wide variety of differingresource items including local software applications and individualdocuments as well as remote services. However, as illustrated by theabove-described ghost services, the described forms of apparatus canalso be used to present items that are not simply place-holders forunderlying services but provide useful information in their own right.

1. An audio user-interfacing method in which items are represented in anaudio field by corresponding synthesized sound sources from where soundsrelated to the items appear to emanate, the method comprising the stepsof: (a) determining, for each said sound source, an associated renderingposition at which the sound source is normally to be synthesized tosound in the audio field; (b) modifying said rendering positions of thesound sources to angularly dilating dilate a region of the audio fieldset relative to a focus reference by modifying and to adjust saidrendering positions of some of the sound sources outside said region toaccommodate that region; and (c) generating, using audio output devices,an audio field in which said sound sources, including those outside saidregion as well as inside, are synthesized at their associated modifiedrendering positions after any modification effected by step (b).
 2. Amethod according to claim 1, wherein the focus reference is movedrelative to the audio field thereby changing which sound sources arepresent in the region of the audio field subject to angular dilation. 3.A method according to claim 2, wherein an audio cursor is movable underuser control in the audio field, the audio cursor serving as said focusreference.
 4. A method according to claim 2, wherein the focus referenceis the current direction of facing of the user as applied to a referenceposition where the user is at least notionally located.
 5. A methodaccording to claim 1, wherein said region is angularly dilated at leastin one of azimuth and elevation.
 6. A method according to claim 1,wherein at least a portion of the audio field outside of said region issubject to angular compression in step (b) to compensate for the angulardilation of said region.
 7. A method according to claim 1, wherein thesound sources in the audio field outside of said region have theirrendering positions adjusted to accommodate that region without therebybeing moved closer to one another.
 8. A method according to claim 1,wherein step (b) can be selectively by-passed under user control.
 9. Amethod according to claim 1, wherein the sound sources are organisedinto groups and step (b) is applied to sounds sources in less than allof said groups.
 10. A method according to claim 1, wherein the angulardilation applied to said region is uniform across the region.
 11. Amethod according to claim 1, wherein the angular dilation applied tosaid region is non-uniform across the region, being greatest at itscentre.
 12. A method according to claim 1, including the step ofadjusting the extent of said region under user control.
 13. A methodaccording to claim 1, wherein the audio field is stabilised relative toone of: a user's head; a user's body; a vehicle in which the user istravelling; and the world, this stabilisation taking account of whetherthe audio output devices are world, vehicle, body or head mounted and,as appropriate, rotation of the user's head or body, or turning of thevehicle.
 14. A method according to claim 1, wherein at least some ofsaid items represented by the sound sources are audio labels forservices, the method further involving selecting a service by selectingthe corresponding audio-label sound source.
 15. An audiouser-interfacing method in which items are represented in an audio fieldby corresponding synthesized sound sources from where sounds related tothe items appear to emanate, the method comprising the steps of: (a)determining for each sound source a rendering position at which thesound source is to be synthesized to sound in the audio field, thesepositions being such that the audio field is angularly dilated in aregion of the audio field set relative to a focus reference; (b)generating an audio field in which said sound sources, including thoseoutside said region as well as inside, are synthesized at theirassociated rendering positions; (c) moving the focus reference relativeto the audio field thereby changing which sound sources are present inthe region of the audio field subject to angular dilation.
 16. A methodaccording to claim 15, wherein an audio cursor is movable under usercontrol in the audio field, the audio cursor serving as said focusreference.
 17. A method according to claim 15, wherein the focusreference is the current direction of facing of the user as applied to areference position where the user is at least notionally located.
 18. Amethod according to claim 15, wherein said region is angularly dilatedat least in azimuth relative to the user.
 19. A method according toclaim 15, wherein at least a portion of the audio field outside of saidregion is subject to angular compression in step (b) to compensate forthe angular dilation of said region.
 20. A method according to claim 15,wherein the sound sources in the audio field outside of said region havetheir rendering positions adjusted to accommodate that region withoutthereby being moved closer to one another.
 21. Apparatus for providingan audio user interface in which items are represented in an audio fieldby corresponding synthesized sound sources from where sounds related tothe items appear to emanate, the apparatus comprising:rendering-position determining means for determining, for each saidsound source, an associated rendering position at which the sound sourceis to be synthesized to sound in the audio field; position-modifyingmeans for modifying said rendering positions of at least some of thesound sources to angularly dilate a region of the audio field setrelative to a focus reference and to adjust the rendering positions ofsound sources outside said region to accommodate that region; andrendering means, including audio output devices, for generating an audiofield in which said sound sources, including those outside said regionas well as inside, are synthesized at their associated renderingpositions, as modified, after any modification by the position-modifyingmeans, to provide sounds related to the items concerned.
 22. Apparatusaccording to claim 21, wherein the rendering-position determining meanscomprises: means for setting the location of each said sound sourcerelative to an audio-field reference; means for controlling an offsetbetween the audio field reference and a presentation reference, thepresentation reference being determined by a mounting configuration ofthe audio output devices; and means for deriving the rendering positionof each sound source based on the location of the sound source in theaudio field and said offset.
 23. Apparatus according to claim 22,wherein the position-modifying means comprises: means for determiningthe extent of said region, prior to angular dilation, relative to saidpresentation reference; means for comparing the rendering position ofeach sound source with the determined extent of said region whereby todetermine which sound sources lie within the region; means for modifyingrendering positions of the sound sources lying within said region toangularly dilate the audio field in that region; and means for modifyingthe rendering positions of at least some of the sound sources outside ofsaid region to accommodate the region in the audio field.
 24. Apparatusaccording to claim 21, further comprising means for generating an audiocursor movable under user control in the audio field, and means forproviding the position-modifying means with the position of the audiocursor, the position-modifying means being arranged to use the positionof the audio cursor as said focus reference.
 25. Apparatus according toclaim 21, further comprising means for determining the direction offacing of the user, and for passing this direction to theposition-modifying means, the position-modifying means being arranged touse the direction of facing of the user as said focus reference. 26.Apparatus according to claim 21, wherein the position-modifying means isarranged to modify the rendering positions of at least some of the soundsources outside of said region such that at least a portion of the audiofield outside of said region is subject to angular compression. 27.Apparatus according to claim 21, wherein the position-modifying means isarranged to modify the rendering positions of the sound sources outsideof said region such as to accommodate that region without thereby movingthe sources outside of the region closer to one another.
 28. Apparatusaccording to claim 21, wherein the angular dilation applied to saidregion by the position-modifying means is uniform across the region. 29.Apparatus according to claim 21, wherein the angular dilation applied tosaid region by the position-modifying means is non-uniform across theregion, being greatest at its centre.
 30. Apparatus according to claim21, including user-operable means for adjusting the extent of saidregion.
 31. Apparatus according to claim 21, wherein at least some ofsaid items represented by the sound sources are audio labels forservices, the apparatus including a selection arrangement for selectinga service by selecting the corresponding audio-label sound source. 32.Apparatus according to claim 22, wherein said means for setting anoffset between the audio field reference and a presentation reference,comprises user input means for enabling a user to change said offset,and stabilisation means for varying said offset such as to stabilise theaudio field reference relative to one of: a user's head; a user's body;a vehicle mounting the apparatus; and the world.
 33. Apparatus forproviding an audio user interface in which items are represented in anaudio field by corresponding synthesized sound sources from where soundsrelated to the items appear to emanate, the apparatus comprising: arendering-position determining arrangement arranged to determine, foreach said sound source, an associated rendering position at which thesound source is to be synthesized to sound in the audio field; aposition-modifying arrangement arranged to modify said renderingpositions of some of the sound sources to angularly dilate a region ofthe audio field set relative to a focus reference and to adjust therendering positions of sound sources outside said region to accommodatethat region; and a rendering subsystem, including audio output devices,arranged to generate an audio field in which said sound sources,including those outside said region as well as inside, are synthesizedat their associated rendering positions, after any modification asmodified, by the position-modifying arrangement, to provide soundsrelated to the items concerned.
 34. Apparatus according to claim 33,wherein the rendering-position determining arrangement comprises: asetting arrangement for setting the location of each said sound sourcerelative to an audio-field reference; a control arrangement forcontrolling an offset between the audio field reference and apresentation reference, the presentation reference being determined by amounting configuration of the audio output devices; and a derivingarrangement arranged to derive the rendering position of each soundsource based on the location of the sound source in the audio field andsaid offset.
 35. Apparatus according to claim 34, wherein theposition-modifying arrangement comprises: an arrangement arranged todetermine the extent of said region, prior to angular dilation, relativeto said presentation reference; a comparator for comparing the renderingposition of each sound source with the determined extent of said regionwhereby to determine which sound sources lie within the region; anarrangement arranged to modify rendering positions of the sound sourceslying within said region to angularly dilate the audio field in thatregion; and an arrangement arranged to modify the rendering positions ofat least some of the sound sources outside of said region to accommodatethe region in the audio field.
 36. Apparatus according to claim 33,further comprising a cursor subsystem operative to generate an audiocursor movable under user control in the audio field, and an arrangementoperative to provide the position-modifying arrangement with theposition of the audio cursor, the position-modifying arrangement beingarranged to use the position of the audio cursor as said focusreference.
 37. Apparatus according to claim 33, further comprising anarrangement arranged to determine the direction of facing of the user,and to pass this direction to the position-modifying arrangement, theposition-modifying arrangement being arranged to use the direction offacing of the user as said focus reference.
 38. Apparatus according toclaim 33, wherein the position-modifying arrangement is arranged tomodify the rendering positions of at least some of the sound sourcesoutside of said region such that at least a portion of the audio fieldoutside of said region is subject to angular compression.
 39. Apparatusaccording to claim 33, wherein the position-modifying arrangement isarranged to modify the rendering positions of the sound sources outsideof said region such as to accommodate that region without thereby movingthe sources outside of the region closer to one another.
 40. Apparatusaccording to claim 33, wherein the angular dilation applied to saidregion by the position-modifying arrangement is uniform across theregion.
 41. Apparatus according to claim 33, wherein the angulardilation applied to said region by the position-modifying an arrangementis non-uniform across the region, being greatest at its centre. 42.Apparatus according to claim 33, including user-operable functionalityfor adjusting the extent of said region.
 43. Apparatus according toclaim 33, wherein at least some of said items represented by the soundsources are audio labels for services, the apparatus including aselection arrangement for selecting a service by selecting thecorresponding audio-label sound source.
 44. Apparatus according to claim34, wherein the control arrangement comprises user input functionalityfor enabling a user to change said offset, and a stabilisationarrangement arranged to vary said offset such as to stabilise the audiofield reference relative to one of: a user's head; a user's body; avehicle mounting the apparatus; and the world.
 45. Apparatus forproviding an audio user interface in which items are represented in anaudio field by corresponding synthesized sound sources from where soundsrelated to the items appear to emanate, the apparatus comprising: arendering-position determination arrangement arranged to determine foreach sound source a rendering position at which the sound source is tobe synthesized to sound in the audio field, these positions being suchthat the audio field is angularly dilated in a region of the audio fieldset relative to a focus reference; a rendering subsystem, includingaudio output devices, arranged to generate an audio field in which saidsound sources, including those outside said region as well as inside,are synthesized at their associated rendering positions; and inputfunctionality for moving the focus reference relative to the audio fieldthereby changing which sound sources are present in the region of theaudio field subject to angular dilation.
 46. An audio user-interfacingmethod in which items are represented in an audio field by correspondingsynthesized sound sources from where sounds related to the items appearto emanate, the method comprising the steps of: (a) determining, foreach said sound source, an associated rendering position at which thesound source is normally to be synthesized to sound in the audio field;(b) angularly dilating a region of the audio field set relative to afocus reference by modifying said rendering positions of at least someof the sound sources; (c) generating, using audio output devices, anaudio field in which said sound sources are synthesized at theirassociated rendering positions after any modification effected bystep(b); and (d) compensating for the angular dilation of said region bysubjecting, during step (b), at least a portion of the audio fieldoutside of said region to angular compression.
 47. An audiouser-interfacing method in which items are represented in an audio fieldby corresponding synthesized sound sources from where sounds related tothe items appear to emanate, the method comprising the steps of: (a)determining for each sound source a rendering position at which thesound source is to be synthesized to sound in the audio field, thesepositions being such that the audio field is angular dilated in a regionof the audio field set relative to a focus reference; (b) compensatingfor the angular dilation of said region by generating an audio fieldoutside of said region in which said sound sources are synthesized attheir associated rendering positions; and (c) moving the focus referencerelative to the audio field thereby changing which sound sources arepresent in the region of the audio field subject to angular dilation.48. Apparatus for providing an audio user interface in which items arerepresented in an audio field by corresponding synthesized sound sourcesfrom where sounds related to the items appear to emanate, the apparatuscomprising: rendering-position determining means for determining, foreach said sound source, an associated rendering position at which thesound source is to be synthesized to sound in the audio field;position-modifying means for modifying said rendering positions of atleast some of the sound sources to angularly dilate a region of theaudio field set relative to a focus reference, the position-modifyingmeans being arranged to modify the rendering positions of at least someof the sound sources outside of said region such that at least a portionof the audio field outside of said region is subject to angularcompression; and rendering means, including audio output devices, forgenerating an audio field in which said sound sources are synthesized attheir associated rendering positions, after any modification by theposition-modifying means, to provide sounds related to the itemsconcerned.
 49. Apparatus for providing an audio user interface in whichitems are represented in an audio field by corresponding synthesizedsound sources from where sounds related to the items appear to emanate,the apparatus comprising: a rendering-position determining arrangementarranged to determine, for each said sound source, an associatedrendering position at which the sound source is to be synthesized tosound in the audio field; a position-modifying arrangement arranged tomodify said rendering positions of at least some of the sound sources toangularly dilate a region of the audio field set relative to a focusreference, the position-modifying arrangement being arranged to modifythe rendering positions of at least some of the sound sources outside ofsaid region such that at least a portion of the audio field outside ofsaid region is subject to angular compression; and a renderingsubsystem, including audio output devices, arranged to generate an audiofield in which said sound sources are synthesized at their associatedrendering positions, after any modification by the position-modifyingarrangement, to provide sounds related to the items concerned.